2021-03-11T00:36:05.367 INFO:root:teuthology version: 1.1.0-b6eecfe8 2021-03-11T00:36:05.369 DEBUG:teuthology.run:Teuthology command: teuthology --name sage-2021-03-11_00:22:56-upgrade:octopus-x-wip-sage-testing-2021-03-10-1754-distro-basic-smithi --description upgrade:octopus-x/rgw-multisite/{clusters frontend overrides realm tasks upgrade/secondary} --verbose --archive /home/teuthworker/archive/sage-2021-03-11_00:22:56-upgrade:octopus-x-wip-sage-testing-2021-03-10-1754-distro-basic-smithi/5954563 --owner scheduled_sage@teuthology -- /home/teuthworker/archive/sage-2021-03-11_00:22:56-upgrade:octopus-x-wip-sage-testing-2021-03-10-1754-distro-basic-smithi/5954563/orig.config.yaml 2021-03-11T00:36:05.423 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2021-03-11T00:36:05.493 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/sage-2021-03-11_00:22:56-upgrade:octopus-x-wip-sage-testing-2021-03-10-1754-distro-basic-smithi/5954563 branch: wip-sage-testing-2021-03-10-1754 description: upgrade:octopus-x/rgw-multisite/{clusters frontend overrides realm tasks upgrade/secondary} email: null first_in_suite: false job_id: '5954563' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: sage-2021-03-11_00:22:56-upgrade:octopus-x-wip-sage-testing-2021-03-10-1754-distro-basic-smithi nuke-on-error: true overrides: admin_socket: branch: wip-sage-testing-2021-03-10-1754 ceph: conf: client: debug rgw: 20 rgw crypt require ssl: false rgw crypt s3 kms backend: testing rgw crypt s3 kms encryption keys: testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= rgw curl low speed time: 300 rgw data log num shards: 4 rgw md log max shards: 4 rgw sync log trim interval: 0 rgw sync obj etag verify: true setgroup: ceph setuser: ceph mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug filestore: 20 debug journal: 20 debug ms: 1 debug osd: 25 log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(PG_AVAILABILITY\) log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 409dc4b843da18e60e08a06ad16195db022fb227 wait-for-scrub: false ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: branch: octopus sha1: 409dc4b843da18e60e08a06ad16195db022fb227 rgw: compression type: random frontend: beast rgw-multisite: realm: is default: true name: test-realm zonegroups: - endpoints: - c1.client.0 is_default: true is_master: true name: test-zonegroup zones: - endpoints: - c1.client.0 is_default: true is_master: true name: test-zone1 - endpoints: - c2.client.0 is_default: true name: test-zone2 - endpoints: - c2.client.1 is_pubsub: true name: test-zone3 rgw-multisite-tests: args: - tests.py - tests_ps.py branch: octopus repo: https://github.com/ceph/ceph.git workunit: branch: wip-sage-testing-2021-03-10-1754 sha1: 409dc4b843da18e60e08a06ad16195db022fb227 owner: scheduled_sage@teuthology priority: 70 repo: git://git.ceph.com/ceph-ci.git roles: - - c1.mon.a - c1.mgr.x - c1.osd.0 - c1.osd.1 - c1.osd.2 - c1.client.0 - - c2.mon.a - c2.mgr.x - c2.osd.0 - c2.osd.1 - c2.osd.2 - c2.client.0 - c2.client.1 sha1: 409dc4b843da18e60e08a06ad16195db022fb227 sleep_before_teardown: 0 suite: upgrade:octopus-x suite_branch: wip-sage-testing-2021-03-10-1754 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_409dc4b843da18e60e08a06ad16195db022fb227/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph-ci.git suite_sha1: 409dc4b843da18e60e08a06ad16195db022fb227 targets: smithi119.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBkw+7z94RmG19ReztArI3vjOPfFJkjaSGO5pvROxpV9zmO+L2Yys46XUKBb++5odwXIDVu7eHgCOrPnKsDbyd7VNJxGSk0qsuK0++1L8XQGrBDTjoFesokochACASxPO+9Ck7rwQz334ECXeJFEPuroH0UzS3GDGN2inhQ5VOUJMX5dSJ5o8pZ5o+1O89zprteWvLTh5RHvROH1LN+UHQUMMc97Z8tT3eh6OB6YB6S+OtTgtOPwvKV6nbBa43abd3qIVXbgLsMgd4AKkW9Su4QFL7JmGZqUyKNTVigVHiIsaQLA4K3OTAZtndROLdkKtEOziGkPAhdW7wJ6ZtEPU9 smithi124.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZAjNr5vAVlml6ZWyM0iMPm0+6NnbBxASsDjHySfZnRlfgNIjsgDdGmp+xM6cJykurCcFvTrSR6jGNZQ/SVZPriuysfAM4a97RSpxADK3ZIaF6Ngt13G6ltPTCpnDHjeIV+79rQb4R3WfxjXlYQLpHqmX30QKEF1Jttoy0zsmW5RDn47fnAKDoz8GtcV2q0pLtijDntA20Mra53SqsHom6N4AHW111a5xzx1fxB1KV0NvivusA+qdStJPIuEJh7zxwOyVrhySOQd+cmbGpC1oAS8sL5+Fq8KsPLnPncdPfxxmTjWm5+wn+02475zvLA/hmXXKCJXp8QytyAIXOk0c1 tasks: - install: null - ceph: cluster: c1 - ceph: cluster: c2 - parallel: - upgrade-task - rgw: c1.client.0: port: 8000 c2.client.0: port: 8000 c2.client.1: port: 8001 - rgw-multisite: null - rgw-multisite-tests: config: reconfigure_delay: 60 teuthology_branch: master teuthology_sha1: b6eecfe89a4363750d696160843a096384b357a8 timestamp: 2021-03-11_00:22:56 tube: smithi upgrade-task: - install.upgrade: c2.client.0: null c2.client.1: null c2.mgr.x: null c2.mon.a: null c2.osd.0: null c2.osd.1: null c2.osd.2: null user: sage verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2137 2021-03-11T00:36:05.494 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_409dc4b843da18e60e08a06ad16195db022fb227/qa; will attempt to use it 2021-03-11T00:36:05.508 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_409dc4b843da18e60e08a06ad16195db022fb227/qa/tasks 2021-03-11T00:36:05.509 INFO:teuthology.run_tasks:Running task internal.check_packages... 2021-03-11T00:36:05.511 INFO:teuthology.task.internal:Checking packages... 2021-03-11T00:36:05.512 INFO:teuthology.task.internal:Checking packages skipped, missing os_type 'None' or ceph hash '409dc4b843da18e60e08a06ad16195db022fb227' 2021-03-11T00:36:05.512 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2021-03-11T00:36:05.518 INFO:teuthology.task.internal:no buildpackages task found 2021-03-11T00:36:05.519 INFO:teuthology.run_tasks:Running task internal.save_config... 2021-03-11T00:36:05.534 INFO:teuthology.task.internal:Saving configuration 2021-03-11T00:36:05.564 INFO:teuthology.run_tasks:Running task internal.check_lock... 2021-03-11T00:36:05.586 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-11T00:36:05.610 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2021-03-11 00:30:49.990215', 'description': '/home/teuthworker/archive/sage-2021-03-11_00:22:56-upgrade:octopus-x-wip-sage-testing-2021-03-10-1754-distro-basic-smithi/5954563', 'locked_by': 'scheduled_sage@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIC8GXRYZwScBZrNW7fHpLc7iFrxxqehvahwAlIGNo0kM', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi119.front.sepia.ceph.com'} 2021-03-11T00:36:05.630 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2021-03-11 00:30:49.990194', 'description': '/home/teuthworker/archive/sage-2021-03-11_00:22:56-upgrade:octopus-x-wip-sage-testing-2021-03-10-1754-distro-basic-smithi/5954563', 'locked_by': 'scheduled_sage@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICvoCDyxfk9ougn9GZjGr90xuCMJ9pvwp27/NJoicPv7', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi124.front.sepia.ceph.com'} 2021-03-11T00:36:05.631 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2021-03-11T00:36:05.637 INFO:teuthology.task.internal:roles: ubuntu@smithi119.front.sepia.ceph.com - ['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0'] 2021-03-11T00:36:05.638 INFO:teuthology.task.internal:roles: ubuntu@smithi124.front.sepia.ceph.com - ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1'] 2021-03-11T00:36:05.638 INFO:teuthology.run_tasks:Running task console_log... 2021-03-11T00:36:05.800 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff07dcf4950>, signals=[15]) 2021-03-11T00:36:05.801 INFO:teuthology.run_tasks:Running task internal.connect... 2021-03-11T00:36:05.808 INFO:teuthology.task.internal:Opening connections... 2021-03-11T00:36:05.809 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi119.front.sepia.ceph.com 2021-03-11T00:36:05.811 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-03-11T00:36:05.889 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi124.front.sepia.ceph.com 2021-03-11T00:36:05.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-03-11T00:36:05.967 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2021-03-11T00:36:05.974 DEBUG:teuthology.orchestra.run.smithi119:> uname -m 2021-03-11T00:36:06.092 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64 2021-03-11T00:36:06.093 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release 2021-03-11T00:36:06.139 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="Ubuntu" 2021-03-11T00:36:06.140 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2021-03-11T00:36:06.140 INFO:teuthology.orchestra.run.smithi119.stdout:ID=ubuntu 2021-03-11T00:36:06.140 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE=debian 2021-03-11T00:36:06.140 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2021-03-11T00:36:06.140 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="18.04" 2021-03-11T00:36:06.141 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://www.ubuntu.com/" 2021-03-11T00:36:06.141 INFO:teuthology.orchestra.run.smithi119.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-03-11T00:36:06.141 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-03-11T00:36:06.141 INFO:teuthology.orchestra.run.smithi119.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-03-11T00:36:06.142 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_CODENAME=bionic 2021-03-11T00:36:06.142 INFO:teuthology.orchestra.run.smithi119.stdout:UBUNTU_CODENAME=bionic 2021-03-11T00:36:06.143 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2021-03-11T00:36:06.163 DEBUG:teuthology.orchestra.run.smithi124:> uname -m 2021-03-11T00:36:06.170 INFO:teuthology.orchestra.run.smithi124.stdout:x86_64 2021-03-11T00:36:06.170 DEBUG:teuthology.orchestra.run.smithi124:> cat /etc/os-release 2021-03-11T00:36:06.219 INFO:teuthology.orchestra.run.smithi124.stdout:NAME="Ubuntu" 2021-03-11T00:36:06.220 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2021-03-11T00:36:06.220 INFO:teuthology.orchestra.run.smithi124.stdout:ID=ubuntu 2021-03-11T00:36:06.220 INFO:teuthology.orchestra.run.smithi124.stdout:ID_LIKE=debian 2021-03-11T00:36:06.220 INFO:teuthology.orchestra.run.smithi124.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2021-03-11T00:36:06.221 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION_ID="18.04" 2021-03-11T00:36:06.221 INFO:teuthology.orchestra.run.smithi124.stdout:HOME_URL="https://www.ubuntu.com/" 2021-03-11T00:36:06.221 INFO:teuthology.orchestra.run.smithi124.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-03-11T00:36:06.221 INFO:teuthology.orchestra.run.smithi124.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-03-11T00:36:06.222 INFO:teuthology.orchestra.run.smithi124.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-03-11T00:36:06.222 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION_CODENAME=bionic 2021-03-11T00:36:06.222 INFO:teuthology.orchestra.run.smithi124.stdout:UBUNTU_CODENAME=bionic 2021-03-11T00:36:06.223 INFO:teuthology.lock.ops:Updating smithi124.front.sepia.ceph.com on lock server 2021-03-11T00:36:06.240 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2021-03-11T00:36:06.258 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2021-03-11T00:36:06.277 INFO:teuthology.task.internal:Checking for old test directory... 2021-03-11T00:36:06.278 DEBUG:teuthology.orchestra.run.smithi119:> test '!' -e /home/ubuntu/cephtest 2021-03-11T00:36:06.281 DEBUG:teuthology.orchestra.run.smithi124:> test '!' -e /home/ubuntu/cephtest 2021-03-11T00:36:06.287 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2021-03-11T00:36:06.310 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2021-03-11T00:36:06.310 DEBUG:teuthology.orchestra.run.smithi119:> test -z $(ls -A /var/lib/ceph) 2021-03-11T00:36:06.327 DEBUG:teuthology.orchestra.run.smithi124:> test -z $(ls -A /var/lib/ceph) 2021-03-11T00:36:06.350 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2021-03-11T00:36:06.406 INFO:teuthology.run_tasks:Running task kernel... 2021-03-11T00:36:06.438 DEBUG:teuthology.task.kernel:normalized config {'c1.mon.a': {'kdb': True, 'sha1': 'distro'}, 'c1.mgr.x': {'kdb': True, 'sha1': 'distro'}, 'c1.osd.0': {'kdb': True, 'sha1': 'distro'}, 'c1.osd.1': {'kdb': True, 'sha1': 'distro'}, 'c1.osd.2': {'kdb': True, 'sha1': 'distro'}, 'c1.client.0': {'kdb': True, 'sha1': 'distro'}, 'c2.mon.a': {'kdb': True, 'sha1': 'distro'}, 'c2.mgr.x': {'kdb': True, 'sha1': 'distro'}, 'c2.osd.0': {'kdb': True, 'sha1': 'distro'}, 'c2.osd.1': {'kdb': True, 'sha1': 'distro'}, 'c2.osd.2': {'kdb': True, 'sha1': 'distro'}, 'c2.client.0': {'kdb': True, 'sha1': 'distro'}, 'c2.client.1': {'kdb': True, 'sha1': 'distro'}} 2021-03-11T00:36:06.452 INFO:teuthology.task.kernel:config {'c1.mon.a': {'kdb': True, 'sha1': 'distro'}, 'c2.mon.a': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2021-03-11T00:36:06.453 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2021-03-11T00:36:06.471 INFO:teuthology.orchestra.run.smithi119.stdout:4.15.0-136-generic 2021-03-11T00:36:06.472 INFO:teuthology.task.kernel:Running kernel on smithi119: 4.15.0-136-generic 2021-03-11T00:36:06.472 DEBUG:teuthology.orchestra.run.smithi119:> sudo apt-get clean 2021-03-11T00:36:06.580 DEBUG:teuthology.orchestra.run.smithi119:> sudo apt-get update 2021-03-11T00:36:06.858 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-03-11T00:36:06.859 INFO:teuthology.orchestra.run.smithi119.stdout:Get:2 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2021-03-11T00:36:06.920 INFO:teuthology.orchestra.run.smithi119.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2021-03-11T00:36:07.238 INFO:teuthology.orchestra.run.smithi119.stdout:Get:4 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [926 kB] 2021-03-11T00:36:07.247 INFO:teuthology.orchestra.run.smithi119.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2021-03-11T00:36:07.423 INFO:teuthology.orchestra.run.smithi119.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,228 kB] 2021-03-11T00:36:07.602 INFO:teuthology.orchestra.run.smithi119.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,573 kB] 2021-03-11T00:36:07.712 INFO:teuthology.orchestra.run.smithi119.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [302 kB] 2021-03-11T00:36:07.729 INFO:teuthology.orchestra.run.smithi119.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [245 kB] 2021-03-11T00:36:07.740 INFO:teuthology.orchestra.run.smithi119.stdout:Get:10 http://security.ubuntu.com/ubuntu bionic-security/restricted i386 Packages [17.7 kB] 2021-03-11T00:36:07.740 INFO:teuthology.orchestra.run.smithi119.stdout:Get:11 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [31.7 kB] 2021-03-11T00:36:07.741 INFO:teuthology.orchestra.run.smithi119.stdout:Get:12 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [979 kB] 2021-03-11T00:36:07.775 INFO:teuthology.orchestra.run.smithi119.stdout:Get:13 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,113 kB] 2021-03-11T00:36:07.804 INFO:teuthology.orchestra.run.smithi119.stdout:Get:14 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [249 kB] 2021-03-11T00:36:07.810 INFO:teuthology.orchestra.run.smithi119.stdout:Get:15 http://security.ubuntu.com/ubuntu bionic-security/multiverse i386 Packages [6,480 B] 2021-03-11T00:36:07.810 INFO:teuthology.orchestra.run.smithi119.stdout:Get:16 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [19.1 kB] 2021-03-11T00:36:07.811 INFO:teuthology.orchestra.run.smithi119.stdout:Get:17 http://security.ubuntu.com/ubuntu bionic-security/multiverse Translation-en [4,412 B] 2021-03-11T00:36:07.891 INFO:teuthology.orchestra.run.smithi119.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,913 kB] 2021-03-11T00:36:08.042 INFO:teuthology.orchestra.run.smithi119.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [266 kB] 2021-03-11T00:36:08.053 INFO:teuthology.orchestra.run.smithi119.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/restricted i386 Packages [24.3 kB] 2021-03-11T00:36:08.054 INFO:teuthology.orchestra.run.smithi119.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [35.3 kB] 2021-03-11T00:36:08.055 INFO:teuthology.orchestra.run.smithi119.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,561 kB] 2021-03-11T00:36:08.119 INFO:teuthology.orchestra.run.smithi119.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,719 kB] 2021-03-11T00:36:08.155 INFO:teuthology.orchestra.run.smithi119.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [24.9 kB] 2021-03-11T00:36:08.155 INFO:teuthology.orchestra.run.smithi119.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse i386 Packages [11.6 kB] 2021-03-11T00:36:08.155 INFO:teuthology.orchestra.run.smithi119.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [6,464 B] 2021-03-11T00:36:09.267 INFO:teuthology.orchestra.run.smithi119.stdout:Fetched 12.5 MB in 2s (5,047 kB/s) 2021-03-11T00:36:10.735 INFO:teuthology.orchestra.run.smithi119.stdout:Reading package lists... 2021-03-11T00:36:10.782 DEBUG:teuthology.orchestra.run.smithi119:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-03-11T00:36:10.866 INFO:teuthology.orchestra.run.smithi119.stdout:Reading package lists... 2021-03-11T00:36:11.123 INFO:teuthology.orchestra.run.smithi119.stdout:Building dependency tree... 2021-03-11T00:36:11.124 INFO:teuthology.orchestra.run.smithi119.stdout:Reading state information... 2021-03-11T00:36:11.203 INFO:teuthology.orchestra.run.smithi119.stderr:E: Unable to locate package linux-image-current-generic 2021-03-11T00:36:11.205 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-03-11T00:36:11.206 DEBUG:teuthology.orchestra.run.smithi119:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-03-11T00:36:11.285 INFO:teuthology.orchestra.run.smithi119.stdout:Reading package lists... 2021-03-11T00:36:11.516 INFO:teuthology.orchestra.run.smithi119.stdout:Building dependency tree... 2021-03-11T00:36:11.516 INFO:teuthology.orchestra.run.smithi119.stdout:Reading state information... 2021-03-11T00:36:11.720 INFO:teuthology.orchestra.run.smithi119.stdout:linux-image-generic is already the newest version (4.15.0.136.123). 2021-03-11T00:36:11.720 INFO:teuthology.orchestra.run.smithi119.stdout:linux-image-generic set to manually installed. 2021-03-11T00:36:11.720 INFO:teuthology.orchestra.run.smithi119.stdout:0 upgraded, 0 newly installed, 0 to remove and 17 not upgraded. 2021-03-11T00:36:11.722 DEBUG:teuthology.orchestra.run.smithi119:> dpkg -s linux-image-generic 2021-03-11T00:36:11.745 INFO:teuthology.orchestra.run.smithi119.stdout:Package: linux-image-generic 2021-03-11T00:36:11.746 INFO:teuthology.orchestra.run.smithi119.stdout:Status: install ok installed 2021-03-11T00:36:11.746 INFO:teuthology.orchestra.run.smithi119.stdout:Priority: optional 2021-03-11T00:36:11.746 INFO:teuthology.orchestra.run.smithi119.stdout:Section: kernel 2021-03-11T00:36:11.747 INFO:teuthology.orchestra.run.smithi119.stdout:Installed-Size: 17 2021-03-11T00:36:11.747 INFO:teuthology.orchestra.run.smithi119.stdout:Maintainer: Ubuntu Kernel Team 2021-03-11T00:36:11.747 INFO:teuthology.orchestra.run.smithi119.stdout:Architecture: amd64 2021-03-11T00:36:11.748 INFO:teuthology.orchestra.run.smithi119.stdout:Source: linux-meta 2021-03-11T00:36:11.748 INFO:teuthology.orchestra.run.smithi119.stdout:Version: 4.15.0.136.123 2021-03-11T00:36:11.749 INFO:teuthology.orchestra.run.smithi119.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-03-11T00:36:11.749 INFO:teuthology.orchestra.run.smithi119.stdout:Depends: linux-image-4.15.0-136-generic, linux-modules-extra-4.15.0-136-generic, linux-firmware, intel-microcode, amd64-microcode 2021-03-11T00:36:11.749 INFO:teuthology.orchestra.run.smithi119.stdout:Recommends: thermald 2021-03-11T00:36:11.750 INFO:teuthology.orchestra.run.smithi119.stdout:Description: Generic Linux kernel image 2021-03-11T00:36:11.750 INFO:teuthology.orchestra.run.smithi119.stdout: This package will always depend on the latest generic kernel image 2021-03-11T00:36:11.751 INFO:teuthology.orchestra.run.smithi119.stdout: available. 2021-03-11T00:36:11.751 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2021-03-11T00:36:11.752 DEBUG:teuthology.orchestra.run.smithi124:> uname -r 2021-03-11T00:36:11.761 INFO:teuthology.orchestra.run.smithi124.stdout:4.15.0-136-generic 2021-03-11T00:36:11.762 INFO:teuthology.task.kernel:Running kernel on smithi124: 4.15.0-136-generic 2021-03-11T00:36:11.762 DEBUG:teuthology.orchestra.run.smithi124:> sudo apt-get clean 2021-03-11T00:36:11.870 DEBUG:teuthology.orchestra.run.smithi124:> sudo apt-get update 2021-03-11T00:36:11.981 INFO:teuthology.orchestra.run.smithi124.stdout:Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2021-03-11T00:36:12.141 INFO:teuthology.orchestra.run.smithi124.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-03-11T00:36:12.206 INFO:teuthology.orchestra.run.smithi124.stdout:Get:3 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,573 kB] 2021-03-11T00:36:12.217 INFO:teuthology.orchestra.run.smithi124.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2021-03-11T00:36:12.309 INFO:teuthology.orchestra.run.smithi124.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [926 kB] 2021-03-11T00:36:12.323 INFO:teuthology.orchestra.run.smithi124.stdout:Get:6 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [302 kB] 2021-03-11T00:36:12.326 INFO:teuthology.orchestra.run.smithi124.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/restricted i386 Packages [17.7 kB] 2021-03-11T00:36:12.326 INFO:teuthology.orchestra.run.smithi124.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [245 kB] 2021-03-11T00:36:12.328 INFO:teuthology.orchestra.run.smithi124.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [31.7 kB] 2021-03-11T00:36:12.329 INFO:teuthology.orchestra.run.smithi124.stdout:Get:10 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,113 kB] 2021-03-11T00:36:12.340 INFO:teuthology.orchestra.run.smithi124.stdout:Get:11 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [979 kB] 2021-03-11T00:36:12.350 INFO:teuthology.orchestra.run.smithi124.stdout:Get:12 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [249 kB] 2021-03-11T00:36:12.351 INFO:teuthology.orchestra.run.smithi124.stdout:Get:13 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [19.1 kB] 2021-03-11T00:36:12.351 INFO:teuthology.orchestra.run.smithi124.stdout:Get:14 http://security.ubuntu.com/ubuntu bionic-security/multiverse i386 Packages [6,480 B] 2021-03-11T00:36:12.352 INFO:teuthology.orchestra.run.smithi124.stdout:Get:15 http://security.ubuntu.com/ubuntu bionic-security/multiverse Translation-en [4,412 B] 2021-03-11T00:36:12.558 INFO:teuthology.orchestra.run.smithi124.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2021-03-11T00:36:12.701 INFO:teuthology.orchestra.run.smithi124.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,228 kB] 2021-03-11T00:36:13.023 INFO:teuthology.orchestra.run.smithi124.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,913 kB] 2021-03-11T00:36:13.136 INFO:teuthology.orchestra.run.smithi124.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/restricted i386 Packages [24.3 kB] 2021-03-11T00:36:13.137 INFO:teuthology.orchestra.run.smithi124.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [266 kB] 2021-03-11T00:36:13.142 INFO:teuthology.orchestra.run.smithi124.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [35.3 kB] 2021-03-11T00:36:13.143 INFO:teuthology.orchestra.run.smithi124.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,561 kB] 2021-03-11T00:36:13.233 INFO:teuthology.orchestra.run.smithi124.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,719 kB] 2021-03-11T00:36:13.259 INFO:teuthology.orchestra.run.smithi124.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [24.9 kB] 2021-03-11T00:36:13.259 INFO:teuthology.orchestra.run.smithi124.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse i386 Packages [11.6 kB] 2021-03-11T00:36:13.260 INFO:teuthology.orchestra.run.smithi124.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [6,464 B] 2021-03-11T00:36:13.958 INFO:teuthology.orchestra.run.smithi124.stdout:Fetched 12.5 MB in 2s (6,613 kB/s) 2021-03-11T00:36:15.366 INFO:teuthology.orchestra.run.smithi124.stdout:Reading package lists... 2021-03-11T00:36:15.412 DEBUG:teuthology.orchestra.run.smithi124:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-03-11T00:36:15.490 INFO:teuthology.orchestra.run.smithi124.stdout:Reading package lists... 2021-03-11T00:36:15.722 INFO:teuthology.orchestra.run.smithi124.stdout:Building dependency tree... 2021-03-11T00:36:15.723 INFO:teuthology.orchestra.run.smithi124.stdout:Reading state information... 2021-03-11T00:36:15.803 INFO:teuthology.orchestra.run.smithi124.stderr:E: Unable to locate package linux-image-current-generic 2021-03-11T00:36:15.804 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-03-11T00:36:15.805 DEBUG:teuthology.orchestra.run.smithi124:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-03-11T00:36:15.886 INFO:teuthology.orchestra.run.smithi124.stdout:Reading package lists... 2021-03-11T00:36:16.128 INFO:teuthology.orchestra.run.smithi124.stdout:Building dependency tree... 2021-03-11T00:36:16.129 INFO:teuthology.orchestra.run.smithi124.stdout:Reading state information... 2021-03-11T00:36:16.334 INFO:teuthology.orchestra.run.smithi124.stdout:linux-image-generic is already the newest version (4.15.0.136.123). 2021-03-11T00:36:16.334 INFO:teuthology.orchestra.run.smithi124.stdout:linux-image-generic set to manually installed. 2021-03-11T00:36:16.335 INFO:teuthology.orchestra.run.smithi124.stdout:0 upgraded, 0 newly installed, 0 to remove and 17 not upgraded. 2021-03-11T00:36:16.336 DEBUG:teuthology.orchestra.run.smithi124:> dpkg -s linux-image-generic 2021-03-11T00:36:16.360 INFO:teuthology.orchestra.run.smithi124.stdout:Package: linux-image-generic 2021-03-11T00:36:16.361 INFO:teuthology.orchestra.run.smithi124.stdout:Status: install ok installed 2021-03-11T00:36:16.361 INFO:teuthology.orchestra.run.smithi124.stdout:Priority: optional 2021-03-11T00:36:16.361 INFO:teuthology.orchestra.run.smithi124.stdout:Section: kernel 2021-03-11T00:36:16.361 INFO:teuthology.orchestra.run.smithi124.stdout:Installed-Size: 17 2021-03-11T00:36:16.362 INFO:teuthology.orchestra.run.smithi124.stdout:Maintainer: Ubuntu Kernel Team 2021-03-11T00:36:16.362 INFO:teuthology.orchestra.run.smithi124.stdout:Architecture: amd64 2021-03-11T00:36:16.362 INFO:teuthology.orchestra.run.smithi124.stdout:Source: linux-meta 2021-03-11T00:36:16.362 INFO:teuthology.orchestra.run.smithi124.stdout:Version: 4.15.0.136.123 2021-03-11T00:36:16.362 INFO:teuthology.orchestra.run.smithi124.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-03-11T00:36:16.363 INFO:teuthology.orchestra.run.smithi124.stdout:Depends: linux-image-4.15.0-136-generic, linux-modules-extra-4.15.0-136-generic, linux-firmware, intel-microcode, amd64-microcode 2021-03-11T00:36:16.363 INFO:teuthology.orchestra.run.smithi124.stdout:Recommends: thermald 2021-03-11T00:36:16.363 INFO:teuthology.orchestra.run.smithi124.stdout:Description: Generic Linux kernel image 2021-03-11T00:36:16.363 INFO:teuthology.orchestra.run.smithi124.stdout: This package will always depend on the latest generic kernel image 2021-03-11T00:36:16.364 INFO:teuthology.orchestra.run.smithi124.stdout: available. 2021-03-11T00:36:16.364 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2021-03-11T00:36:16.364 INFO:teuthology.task.kernel:Enabling kdb on c1.mon.a... 2021-03-11T00:36:16.365 DEBUG:teuthology.orchestra.run.smithi119:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-03-11T00:36:16.398 INFO:teuthology.orchestra.run.smithi119.stdout:ttyS1 2021-03-11T00:36:16.420 INFO:teuthology.task.kernel:Enabling kdb on c2.mon.a... 2021-03-11T00:36:16.421 DEBUG:teuthology.orchestra.run.smithi124:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-03-11T00:36:16.453 INFO:teuthology.orchestra.run.smithi124.stdout:ttyS1 2021-03-11T00:36:16.479 INFO:teuthology.run_tasks:Running task internal.base... 2021-03-11T00:36:16.490 INFO:teuthology.task.internal:Creating test directory... 2021-03-11T00:36:16.492 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-03-11T00:36:16.495 DEBUG:teuthology.orchestra.run.smithi124:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-03-11T00:36:16.527 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2021-03-11T00:36:16.535 INFO:teuthology.run_tasks:Running task internal.archive... 2021-03-11T00:36:16.558 INFO:teuthology.task.internal:Creating archive directory... 2021-03-11T00:36:16.559 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-03-11T00:36:16.562 DEBUG:teuthology.orchestra.run.smithi124:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-03-11T00:36:16.591 INFO:teuthology.run_tasks:Running task internal.coredump... 2021-03-11T00:36:16.598 INFO:teuthology.task.internal:Enabling coredump saving... 2021-03-11T00:36:16.599 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-03-11T00:36:16.627 DEBUG:teuthology.orchestra.run.smithi124:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-03-11T00:36:16.647 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-03-11T00:36:16.652 INFO:teuthology.orchestra.run.smithi124.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-03-11T00:36:16.654 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-03-11T00:36:16.658 INFO:teuthology.orchestra.run.smithi124.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-03-11T00:36:16.660 INFO:teuthology.run_tasks:Running task internal.sudo... 2021-03-11T00:36:16.671 INFO:teuthology.task.internal:Configuring sudo... 2021-03-11T00:36:16.671 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-03-11T00:36:16.700 DEBUG:teuthology.orchestra.run.smithi124:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-03-11T00:36:16.716 INFO:teuthology.run_tasks:Running task internal.syslog... 2021-03-11T00:36:16.728 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2021-03-11T00:36:16.728 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-03-11T00:36:16.755 DEBUG:teuthology.orchestra.run.smithi124:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-03-11T00:36:16.763 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-03-11T00:36:16.808 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-03-11T00:36:16.856 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2021-03-11T00:36:16.861 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-03-11T00:36:16.921 DEBUG:teuthology.orchestra.run.smithi124:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-03-11T00:36:16.930 DEBUG:teuthology.orchestra.run.smithi124:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-03-11T00:36:16.980 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2021-03-11T00:36:16.981 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-03-11T00:36:17.041 DEBUG:teuthology.orchestra.run.smithi119:> sudo service rsyslog restart 2021-03-11T00:36:17.044 DEBUG:teuthology.orchestra.run.smithi124:> sudo service rsyslog restart 2021-03-11T00:36:17.126 INFO:teuthology.run_tasks:Running task internal.timer... 2021-03-11T00:36:17.135 INFO:teuthology.task.internal:Starting timer... 2021-03-11T00:36:17.135 INFO:teuthology.run_tasks:Running task pcp... 2021-03-11T00:36:17.156 INFO:teuthology.run_tasks:Running task selinux... 2021-03-11T00:36:17.174 DEBUG:teuthology.task.selinux:Excluding smithi119: OS 'ubuntu' does not support SELinux 2021-03-11T00:36:17.174 DEBUG:teuthology.task.selinux:Excluding smithi124: OS 'ubuntu' does not support SELinux 2021-03-11T00:36:17.175 DEBUG:teuthology.task.selinux:Getting current SELinux state 2021-03-11T00:36:17.175 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2021-03-11T00:36:17.175 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2021-03-11T00:36:17.175 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2021-03-11T00:36:17.190 DEBUG:teuthology.repo_utils:Setting repo remote to git://git.ceph.com/git/ceph-cm-ansible.git 2021-03-11T00:36:17.209 INFO:teuthology.repo_utils:Fetching master from origin 2021-03-11T00:36:17.279 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to origin/master 2021-03-11T00:36:17.400 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2021-03-11T00:36:17.405 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi119.front.sepia.ceph.com,smithi124.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2021-03-11T00:36:18.514 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2021-03-11T00:36:20.860 INFO:teuthology.task.ansible.out:[WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2021-03-11T00:36:23.419 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2021-03-11T00:36:23.471 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:23 +0000 (0:00:00.090) 0:00:00.090 ******** 2021-03-11T00:36:23.472 INFO:teuthology.task.ansible.out: 2021-03-11T00:36:23.491 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:23 +0000 (0:00:00.019) 0:00:00.109 ******** 2021-03-11T00:36:25.197 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2021-03-11T00:36:25.199 INFO:teuthology.task.ansible.out:[DEPRECATION WARNING]: Distribution Ubuntu 18.04 on host smithi119.front.sepia.ceph.com should use /usr/bin/python3, but is using /usr/bin/python for backward compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See https://docs.ansible.com/ansible/2.9/reference_appendices/inter preter_discovery.html for more information. This feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg. ok: [smithi119.front.sepia.ceph.com] 2021-03-11T00:36:25.211 INFO:teuthology.task.ansible.out:[DEPRECATION WARNING]: Distribution Ubuntu 18.04 on host smithi124.front.sepia.ceph.com should use /usr/bin/python3, but is using /usr/bin/python for backward compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See https://docs.ansible.com/ansible/2.9/reference_appendices/inter preter_discovery.html for more information. This feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg. ok: [smithi124.front.sepia.ceph.com] 2021-03-11T00:36:25.285 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:25 +0000 (0:00:01.794) 0:00:01.904 ******** 2021-03-11T00:36:25.309 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:25 +0000 (0:00:00.023) 0:00:01.927 ******** 2021-03-11T00:36:25.600 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2021-03-11T00:36:25.602 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-03-11T00:36:25.627 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:25 +0000 (0:00:00.318) 0:00:02.246 ******** 2021-03-11T00:36:25.659 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:25 +0000 (0:00:00.031) 0:00:02.277 ******** 2021-03-11T00:36:26.430 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2021-03-11T00:36:26.432 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-03-11T00:36:26.456 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:26 +0000 (0:00:00.797) 0:00:03.075 ******** 2021-03-11T00:36:26.579 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-03-11T00:36:26.610 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:26 +0000 (0:00:00.154) 0:00:03.229 ******** 2021-03-11T00:36:26.860 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2021-03-11T00:36:26.862 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.063918", "end": "2021-03-11 00:36:27.093577", "rc": 0, "start": "2021-03-11 00:36:27.029659", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2021-03-11T00:36:26.891 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:26 +0000 (0:00:00.280) 0:00:03.509 ******** 2021-03-11T00:36:26.899 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.073422", "end": "2021-03-11 00:36:26.753551", "rc": 0, "start": "2021-03-11 00:36:26.680129", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2021-03-11T00:36:26.930 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:26 +0000 (0:00:00.038) 0:00:03.548 ******** 2021-03-11T00:36:27.169 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2021-03-11T00:36:27.171 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} ok: [smithi124.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-03-11T00:36:27.196 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:27 +0000 (0:00:00.265) 0:00:03.814 ******** 2021-03-11T00:36:27.217 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:27 +0000 (0:00:00.020) 0:00:03.835 ******** 2021-03-11T00:36:28.117 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2021-03-11T00:36:28.119 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615422987.2329113-14738-169029450571892/source", "state": "file", "uid": 0} 2021-03-11T00:36:28.140 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:28 +0000 (0:00:00.923) 0:00:04.758 ******** 2021-03-11T00:36:28.148 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615422987.2539458-14744-3551122444839/source", "state": "file", "uid": 0} 2021-03-11T00:36:28.173 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:28 +0000 (0:00:00.032) 0:00:04.791 ******** 2021-03-11T00:36:28.623 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2021-03-11T00:36:28.625 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2021-03-11T00:36:28.627 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2021-03-11T00:36:28.648 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2021-03-11T00:36:28.651 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2021-03-11T00:36:28.728 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2021-03-11T00:36:28.732 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2021-03-11T00:36:28.904 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:28 +0000 (0:00:00.731) 0:00:05.522 ******** 2021-03-11T00:36:28.925 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:28 +0000 (0:00:00.021) 0:00:05.543 ******** 2021-03-11T00:36:29.690 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi124.front.sepia.ceph.com] 2021-03-11T00:36:29.700 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] 2021-03-11T00:36:29.775 INFO:teuthology.task.ansible.out:[WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2021-03-11T00:36:29.777 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:29 +0000 (0:00:00.850) 0:00:06.394 ******** 2021-03-11T00:36:29.808 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:29 +0000 (0:00:00.032) 0:00:06.426 ******** 2021-03-11T00:36:30.321 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2021-03-11T00:36:30.323 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.188246", "end": "2021-03-11 00:36:30.291335", "rc": 0, "start": "2021-03-11 00:36:30.103089", "stderr": "", "stderr_lines": [], "stdout": "c31004c4720a55ac12a95d66c32e64f592877604", "stdout_lines": ["c31004c4720a55ac12a95d66c32e64f592877604"]} 2021-03-11T00:36:30.344 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:30 +0000 (0:00:00.535) 0:00:06.962 ******** 2021-03-11T00:36:30.350 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.175679", "end": "2021-03-11 00:36:30.305199", "rc": 0, "start": "2021-03-11 00:36:30.129520", "stderr": "", "stderr_lines": [], "stdout": "c31004c4720a55ac12a95d66c32e64f592877604", "stdout_lines": ["c31004c4720a55ac12a95d66c32e64f592877604"]} 2021-03-11T00:36:30.378 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:30 +0000 (0:00:00.034) 0:00:06.996 ******** 2021-03-11T00:36:30.495 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2021-03-11T00:36:30.496 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.018559", "end": "2021-03-11 00:36:30.372833", "failed_when_result": false, "rc": 0, "start": "2021-03-11 00:36:30.354274", "stderr": "", "stderr_lines": [], "stdout": "df8946f08e870e2d2565b79099289ca48bd825f3", "stdout_lines": ["df8946f08e870e2d2565b79099289ca48bd825f3"]} 2021-03-11T00:36:30.523 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:30 +0000 (0:00:00.145) 0:00:07.142 ******** 2021-03-11T00:36:30.609 INFO:teuthology.task.ansible.out: TASK [Determine if we can skip users and pubkeys updates] ********************** 2021-03-11T00:36:30.611 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:36:30.635 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:30 +0000 (0:00:00.111) 0:00:07.253 ******** 2021-03-11T00:36:30.677 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2021-03-11T00:36:30.679 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.152059", "end": "2021-03-11 00:36:30.910286", "failed_when_result": false, "rc": 0, "start": "2021-03-11 00:36:30.758227", "stderr": "", "stderr_lines": [], "stdout": "df8946f08e870e2d2565b79099289ca48bd825f3", "stdout_lines": ["df8946f08e870e2d2565b79099289ca48bd825f3"]} 2021-03-11T00:36:30.706 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:30 +0000 (0:00:00.071) 0:00:07.324 ******** 2021-03-11T00:36:30.715 INFO:teuthology.task.ansible.out: TASK [Merge extra_admin_users into managed_admin_users] ************************ 2021-03-11T00:36:30.717 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:36:30.743 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:30 +0000 (0:00:00.037) 0:00:07.362 ******** 2021-03-11T00:36:30.771 INFO:teuthology.task.ansible.out: TASK [Determine if we can skip users and pubkeys updates] ********************** 2021-03-11T00:36:30.773 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:36:30.800 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:30 +0000 (0:00:00.056) 0:00:07.418 ******** 2021-03-11T00:36:30.836 INFO:teuthology.task.ansible.out: TASK [Remove managed_admin_users from managed_users] *************************** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:36:30.868 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:30 +0000 (0:00:00.067) 0:00:07.486 ******** 2021-03-11T00:36:30.920 INFO:teuthology.task.ansible.out: 2021-03-11T00:36:30.921 INFO:teuthology.task.ansible.out:TASK [Merge extra_admin_users into managed_admin_users] ************************ skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:36:30.968 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:30 +0000 (0:00:00.100) 0:00:07.587 ******** 2021-03-11T00:36:30.992 INFO:teuthology.task.ansible.out: TASK [Filter the managed_users list] ******************************************* 2021-03-11T00:36:30.994 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:36:31.024 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:31 +0000 (0:00:00.055) 0:00:07.642 ******** 2021-03-11T00:36:31.054 INFO:teuthology.task.ansible.out: TASK [Remove managed_admin_users from managed_users] *************************** skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:36:31.089 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:31 +0000 (0:00:00.065) 0:00:07.707 ******** 2021-03-11T00:36:31.190 INFO:teuthology.task.ansible.out: TASK [Filter the managed_users list] ******************************************* 2021-03-11T00:36:31.195 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [Filter the managed_admin_users list] ************************************* skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:36:31.214 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:31 +0000 (0:00:00.124) 0:00:07.832 ******** 2021-03-11T00:36:31.215 INFO:teuthology.task.ansible.out: 2021-03-11T00:36:31.244 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:31 +0000 (0:00:00.029) 0:00:07.862 ******** 2021-03-11T00:36:31.353 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:36:31.355 INFO:teuthology.task.ansible.out: 2021-03-11T00:36:31.387 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:31 +0000 (0:00:00.142) 0:00:08.005 ******** 2021-03-11T00:36:31.457 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2021-03-11T00:36:31.459 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-03-11T00:36:31.487 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:31 +0000 (0:00:00.099) 0:00:08.105 ******** 2021-03-11T00:36:31.606 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-03-11T00:36:31.637 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:31 +0000 (0:00:00.150) 0:00:08.255 ******** 2021-03-11T00:36:31.824 INFO:teuthology.task.ansible.out: TASK [Create all admin users with sudo access.] ******************************** 2021-03-11T00:36:31.826 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-03-11T00:36:31.922 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-03-11T00:36:31.987 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-03-11T00:36:32.063 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-03-11T00:36:32.128 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-03-11T00:36:32.204 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-03-11T00:36:32.281 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-03-11T00:36:32.344 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-03-11T00:36:32.429 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-03-11T00:36:32.504 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-03-11T00:36:32.568 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-03-11T00:36:32.652 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-03-11T00:36:32.706 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-03-11T00:36:32.791 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-03-11T00:36:32.855 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-03-11T00:36:32.929 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-03-11T00:36:33.003 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-03-11T00:36:33.077 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-03-11T00:36:33.141 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-03-11T00:36:33.216 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-03-11T00:36:33.218 INFO:teuthology.task.ansible.out: 2021-03-11T00:36:33.291 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-03-11T00:36:33.355 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-03-11T00:36:33.430 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-03-11T00:36:33.504 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-03-11T00:36:33.579 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-03-11T00:36:33.644 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-03-11T00:36:33.731 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-03-11T00:36:33.785 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-03-11T00:36:33.871 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-03-11T00:36:33.925 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-03-11T00:36:34.010 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-03-11T00:36:34.074 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-03-11T00:36:34.162 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-03-11T00:36:34.226 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-03-11T00:36:34.300 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2021-03-11T00:36:34.364 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2021-03-11T00:36:34.449 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-03-11T00:36:34.502 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-03-11T00:36:34.504 INFO:teuthology.task.ansible.out: 2021-03-11T00:36:34.598 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-03-11T00:36:34.652 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-03-11T00:36:34.736 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2021-03-11T00:36:34.794 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2021-03-11T00:36:34.880 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1122} 2021-03-11T00:36:34.943 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1122} 2021-03-11T00:36:35.031 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-03-11T00:36:35.095 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-03-11T00:36:35.097 INFO:teuthology.task.ansible.out: 2021-03-11T00:36:35.184 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-03-11T00:36:35.248 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-03-11T00:36:35.354 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-03-11T00:36:35.398 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-03-11T00:36:35.504 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-03-11T00:36:35.557 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-03-11T00:36:35.653 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-03-11T00:36:35.706 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-03-11T00:36:35.793 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-03-11T00:36:35.847 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-03-11T00:36:35.942 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-03-11T00:36:35.986 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-03-11T00:36:36.092 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-03-11T00:36:36.135 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-03-11T00:36:36.231 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-03-11T00:36:36.285 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-03-11T00:36:36.379 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-03-11T00:36:36.434 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-03-11T00:36:36.519 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1133} 2021-03-11T00:36:36.584 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1133} 2021-03-11T00:36:36.671 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-03-11T00:36:36.727 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-03-11T00:36:36.812 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-03-11T00:36:36.865 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-03-11T00:36:37.002 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-03-11T00:36:37.014 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-03-11T00:36:37.143 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1137} 2021-03-11T00:36:37.155 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1137} 2021-03-11T00:36:37.293 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-03-11T00:36:37.294 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-03-11T00:36:37.432 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-03-11T00:36:37.443 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-03-11T00:36:37.581 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-03-11T00:36:37.582 INFO:teuthology.task.ansible.out: 2021-03-11T00:36:37.594 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-03-11T00:36:37.721 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-03-11T00:36:37.733 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-03-11T00:36:37.870 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-03-11T00:36:37.882 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-03-11T00:36:38.020 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-03-11T00:36:38.021 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-03-11T00:36:38.158 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-03-11T00:36:38.169 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-03-11T00:36:38.306 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-03-11T00:36:38.317 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-03-11T00:36:38.455 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-03-11T00:36:38.466 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-03-11T00:36:38.602 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-03-11T00:36:38.603 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-03-11T00:36:38.752 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-03-11T00:36:38.753 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-03-11T00:36:38.890 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-03-11T00:36:38.901 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-03-11T00:36:39.038 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-03-11T00:36:39.040 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-03-11T00:36:39.187 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-03-11T00:36:39.189 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-03-11T00:36:39.327 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-03-11T00:36:39.337 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-03-11T00:36:39.484 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-03-11T00:36:39.485 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-03-11T00:36:39.629 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-03-11T00:36:39.640 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-03-11T00:36:39.776 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-03-11T00:36:39.777 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-03-11T00:36:39.914 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-03-11T00:36:39.915 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-03-11T00:36:40.063 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-03-11T00:36:40.065 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-03-11T00:36:40.206 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-03-11T00:36:40.216 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-03-11T00:36:40.356 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-03-11T00:36:40.368 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-03-11T00:36:40.510 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-03-11T00:36:40.513 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-03-11T00:36:40.513 INFO:teuthology.task.ansible.out: 2021-03-11T00:36:40.663 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-03-11T00:36:40.665 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-03-11T00:36:40.805 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-03-11T00:36:40.817 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-03-11T00:36:40.967 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-03-11T00:36:40.989 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-03-11T00:36:41.116 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-03-11T00:36:41.138 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-03-11T00:36:41.260 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-03-11T00:36:41.283 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-03-11T00:36:41.403 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-03-11T00:36:41.425 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-03-11T00:36:41.552 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-03-11T00:36:41.575 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-03-11T00:36:41.691 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-03-11T00:36:41.713 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-03-11T00:36:41.840 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-03-11T00:36:41.874 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-03-11T00:36:41.980 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-03-11T00:36:42.132 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-03-11T00:36:42.198 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-03-11T00:36:42.294 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-03-11T00:36:42.343 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-03-11T00:36:42.461 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-03-11T00:36:42.494 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-03-11T00:36:42.612 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-03-11T00:36:42.657 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-03-11T00:36:42.753 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-03-11T00:36:42.807 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-03-11T00:36:42.903 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-03-11T00:36:42.946 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-03-11T00:36:43.053 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-03-11T00:36:43.099 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-03-11T00:36:43.248 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-03-11T00:36:43.269 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-03-11T00:36:43.438 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-03-11T00:36:43.472 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-03-11T00:36:43.590 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-03-11T00:36:43.750 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-03-11T00:36:43.783 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-03-11T00:36:43.910 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-03-11T00:36:43.933 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-03-11T00:36:44.060 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-03-11T00:36:44.092 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-03-11T00:36:44.223 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-03-11T00:36:44.245 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-03-11T00:36:44.423 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-03-11T00:36:44.544 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-03-11T00:36:44.576 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1184} 2021-03-11T00:36:44.705 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1184} 2021-03-11T00:36:44.734 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-03-11T00:36:44.845 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-03-11T00:36:44.898 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-03-11T00:36:44.996 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-03-11T00:36:45.039 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-03-11T00:36:45.179 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-03-11T00:36:45.198 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-03-11T00:36:45.355 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-03-11T00:36:45.361 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-03-11T00:36:45.541 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-03-11T00:36:45.553 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-03-11T00:36:45.712 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-03-11T00:36:45.724 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-03-11T00:36:45.863 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-03-11T00:36:45.886 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-03-11T00:36:46.028 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-03-11T00:36:46.030 INFO:teuthology.task.ansible.out: 2021-03-11T00:36:46.066 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-03-11T00:36:46.184 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-03-11T00:36:46.217 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-03-11T00:36:46.372 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-03-11T00:36:46.384 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-03-11T00:36:46.535 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-03-11T00:36:46.537 INFO:teuthology.task.ansible.out: 2021-03-11T00:36:46.569 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-03-11T00:36:46.687 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-03-11T00:36:46.765 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-03-11T00:36:46.927 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-03-11T00:36:46.971 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-03-11T00:36:47.112 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-03-11T00:36:47.158 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-03-11T00:36:47.299 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-03-11T00:36:47.388 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-03-11T00:36:47.474 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-03-11T00:36:47.581 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-03-11T00:36:47.680 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-03-11T00:36:47.768 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-03-11T00:36:47.844 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-03-11T00:36:47.943 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-03-11T00:36:48.009 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-03-11T00:36:48.116 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-03-11T00:36:48.176 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-03-11T00:36:48.272 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-03-11T00:36:48.336 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-03-11T00:36:48.453 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-03-11T00:36:48.527 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-03-11T00:36:48.605 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-03-11T00:36:48.669 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-03-11T00:36:48.745 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-03-11T00:36:48.836 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-03-11T00:36:48.891 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-03-11T00:36:48.987 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-03-11T00:36:49.041 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-03-11T00:36:49.136 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-03-11T00:36:49.189 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-03-11T00:36:49.284 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-03-11T00:36:49.327 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-03-11T00:36:49.423 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-03-11T00:36:49.476 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-03-11T00:36:49.574 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-03-11T00:36:49.627 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-03-11T00:36:49.725 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-03-11T00:36:49.779 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-03-11T00:36:49.865 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-03-11T00:36:49.930 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-03-11T00:36:50.015 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-03-11T00:36:50.068 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-03-11T00:36:50.167 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-03-11T00:36:50.210 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-03-11T00:36:50.317 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-03-11T00:36:50.360 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-03-11T00:36:50.466 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-03-11T00:36:50.510 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-03-11T00:36:50.607 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-03-11T00:36:50.662 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-03-11T00:36:50.759 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-03-11T00:36:50.826 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-03-11T00:36:50.924 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-03-11T00:36:50.979 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-03-11T00:36:51.076 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-03-11T00:36:51.131 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-03-11T00:36:51.228 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-03-11T00:36:51.272 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-03-11T00:36:51.379 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-03-11T00:36:51.435 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-03-11T00:36:51.543 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-03-11T00:36:51.586 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-03-11T00:36:51.687 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-03-11T00:36:51.741 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-03-11T00:36:51.837 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-03-11T00:36:51.880 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-03-11T00:36:51.987 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-03-11T00:36:52.030 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-03-11T00:36:52.127 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-03-11T00:36:52.181 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-03-11T00:36:52.277 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/hmunjulu", "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "move_home": false, "name": "hmunjulu", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-03-11T00:36:52.322 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-03-11T00:36:52.426 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-03-11T00:36:52.480 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/hmunjulu", "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "move_home": false, "name": "hmunjulu", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-03-11T00:36:52.576 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "move_home": false, "name": "pdhange", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-03-11T00:36:52.619 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-03-11T00:36:52.714 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1234, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "move_home": false, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "uid": 1234} 2021-03-11T00:36:52.768 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "move_home": false, "name": "pdhange", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-03-11T00:36:52.866 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1235, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "move_home": false, "name": "adking", "shell": "/bin/bash", "state": "present", "uid": 1235} 2021-03-11T00:36:52.909 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1234, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "move_home": false, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "uid": 1234} 2021-03-11T00:36:53.016 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1236, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "move_home": false, "name": "amathuri", "shell": "/bin/bash", "state": "present", "uid": 1236} 2021-03-11T00:36:53.060 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1235, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "move_home": false, "name": "adking", "shell": "/bin/bash", "state": "present", "uid": 1235} 2021-03-11T00:36:53.168 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1237, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "move_home": false, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "uid": 1237} 2021-03-11T00:36:53.277 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1236, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "move_home": false, "name": "amathuri", "shell": "/bin/bash", "state": "present", "uid": 1236} 2021-03-11T00:36:53.438 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:53 +0000 (0:00:21.800) 0:00:30.056 ******** 2021-03-11T00:36:53.458 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1237, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "move_home": false, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "uid": 1237} 2021-03-11T00:36:53.570 INFO:teuthology.task.ansible.out: TASK [Create all users without sudo access.] *********************************** 2021-03-11T00:36:53.605 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:53 +0000 (0:00:00.167) 0:00:30.223 ******** 2021-03-11T00:36:53.632 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:53 +0000 (0:00:00.026) 0:00:30.250 ******** 2021-03-11T00:36:53.721 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:53 +0000 (0:00:00.088) 0:00:30.339 ******** 2021-03-11T00:36:53.760 INFO:teuthology.task.ansible.out: TASK [Merge managed_users and managed_admin users] ***************************** 2021-03-11T00:36:53.762 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@fedora 2021-03-11T00:36:53.764 INFO:teuthology.task.ansible.out:rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265 2021-03-11T00:36:53.766 INFO:teuthology.task.ansible.out:c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop K 2021-03-11T00:36:53.768 INFO:teuthology.task.ansible.out:D0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9 2021-03-11T00:36:53.770 INFO:teuthology.task.ansible.out:df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 2021-03-11T00:36:53.773 INFO:teuthology.task.ansible.out: 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc9795 2021-03-11T00:36:53.775 INFO:teuthology.task.ansible.out:3cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a 2021-03-11T00:36:53.776 INFO:teuthology.task.ansible.out:4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e 2021-03-11T00:36:53.778 INFO:teuthology.task.ansible.out:7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef14 2021-03-11T00:36:53.780 INFO:teuthology.task.ansible.out:80c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d301 2021-03-11T00:36:53.781 INFO:teuthology.task.ansible.out:7b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214 2021-03-11T00:36:53.784 INFO:teuthology.task.ansible.out:bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelope 2021-03-11T00:36:53.785 INFO:teuthology.task.ansible.out:z@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed 2021-03-11T00:36:53.787 INFO:teuthology.task.ansible.out:19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174f 2021-03-11T00:36:53.788 INFO:teuthology.task.ansible.out:e16d6b872450fe0"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}]}, "changed": false} 2021-03-11T00:36:53.818 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:53 +0000 (0:00:00.097) 0:00:30.436 ******** 2021-03-11T00:36:53.872 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@fedora 2021-03-11T00:36:53.874 INFO:teuthology.task.ansible.out:rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265 2021-03-11T00:36:53.876 INFO:teuthology.task.ansible.out:c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop K 2021-03-11T00:36:53.878 INFO:teuthology.task.ansible.out:D0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9 2021-03-11T00:36:53.880 INFO:teuthology.task.ansible.out:df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 2021-03-11T00:36:53.883 INFO:teuthology.task.ansible.out: 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc9795 2021-03-11T00:36:53.885 INFO:teuthology.task.ansible.out:3cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a 2021-03-11T00:36:53.886 INFO:teuthology.task.ansible.out:4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e 2021-03-11T00:36:53.888 INFO:teuthology.task.ansible.out:7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef14 2021-03-11T00:36:53.889 INFO:teuthology.task.ansible.out:80c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d301 2021-03-11T00:36:53.891 INFO:teuthology.task.ansible.out:7b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214 2021-03-11T00:36:53.893 INFO:teuthology.task.ansible.out:bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelope 2021-03-11T00:36:53.895 INFO:teuthology.task.ansible.out:z@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed 2021-03-11T00:36:53.897 INFO:teuthology.task.ansible.out:19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174f 2021-03-11T00:36:53.898 INFO:teuthology.task.ansible.out:e16d6b872450fe0"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}]}, "changed": false} 2021-03-11T00:36:53.947 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:53 +0000 (0:00:00.129) 0:00:30.565 ******** 2021-03-11T00:36:54.849 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2021-03-11T00:36:54.851 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"after": "c31004c4720a55ac12a95d66c32e64f592877604", "attempts": 1, "before": "c31004c4720a55ac12a95d66c32e64f592877604", "changed": false, "remote_url_changed": false} 2021-03-11T00:36:54.875 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:54 +0000 (0:00:00.927) 0:00:31.493 ******** 2021-03-11T00:36:54.906 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"after": "c31004c4720a55ac12a95d66c32e64f592877604", "attempts": 1, "before": "c31004c4720a55ac12a95d66c32e64f592877604", "changed": false, "remote_url_changed": false} 2021-03-11T00:36:54.959 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:36:54 +0000 (0:00:00.084) 0:00:31.577 ******** 2021-03-11T00:36:55.215 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-03-11T00:36:55.217 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-03-11T00:36:55.360 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-03-11T00:36:55.369 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-03-11T00:36:55.500 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-03-11T00:36:55.522 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-03-11T00:36:55.639 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-03-11T00:36:55.692 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZ 2021-03-11T00:36:55.694 INFO:teuthology.task.ansible.out:tIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2021-03-11T00:36:55.809 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZ 2021-03-11T00:36:55.811 INFO:teuthology.task.ansible.out:tIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2021-03-11T00:36:55.830 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-03-11T00:36:55.948 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-03-11T00:36:55.980 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+ 2021-03-11T00:36:55.983 INFO:teuthology.task.ansible.out:o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-03-11T00:36:56.088 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+ 2021-03-11T00:36:56.091 INFO:teuthology.task.ansible.out:o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-03-11T00:36:56.135 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-03-11T00:36:56.268 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-03-11T00:36:56.302 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-03-11T00:36:56.425 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-03-11T00:36:56.470 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-03-11T00:36:56.482 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:36:56.587 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-03-11T00:36:56.609 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:36:56.621 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMor 2021-03-11T00:36:56.623 INFO:teuthology.task.ansible.out:bnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-03-11T00:36:56.740 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMor 2021-03-11T00:36:56.748 INFO:teuthology.task.ansible.out:bnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-03-11T00:36:56.752 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemer 2021-03-11T00:36:56.754 INFO:teuthology.task.ansible.out:son", "validate_certs": true} skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:36:56.765 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:36:56.881 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemer 2021-03-11T00:36:56.883 INFO:teuthology.task.ansible.out:son", "validate_certs": true} 2021-03-11T00:36:56.892 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:36:56.903 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:36:56.905 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-03-11T00:36:57.044 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-03-11T00:36:57.068 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtS 2021-03-11T00:36:57.070 INFO:teuthology.task.ansible.out:oAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-03-11T00:36:57.180 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtS 2021-03-11T00:36:57.182 INFO:teuthology.task.ansible.out:oAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-03-11T00:36:57.213 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-03-11T00:36:57.331 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-03-11T00:36:57.364 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": 2021-03-11T00:36:57.367 INFO:teuthology.task.ansible.out:null, "state": "present", "user": "davidz", "validate_certs": true} 2021-03-11T00:36:57.470 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": 2021-03-11T00:36:57.472 INFO:teuthology.task.ansible.out:null, "state": "present", "user": "davidz", "validate_certs": true} 2021-03-11T00:36:57.514 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-03-11T00:36:57.602 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-03-11T00:36:57.667 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-03-11T00:36:57.744 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-03-11T00:36:57.750 INFO:teuthology.task.ansible.out: 2021-03-11T00:36:57.810 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2021-03-11T00:36:57.884 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2021-03-11T00:36:57.982 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9 2021-03-11T00:36:57.983 INFO:teuthology.task.ansible.out:CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-03-11T00:36:58.050 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9 2021-03-11T00:36:58.052 INFO:teuthology.task.ansible.out:CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-03-11T00:36:58.164 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBY 2021-03-11T00:36:58.167 INFO:teuthology.task.ansible.out:B0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-03-11T00:36:58.203 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBY 2021-03-11T00:36:58.205 INFO:teuthology.task.ansible.out:B0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-03-11T00:36:58.351 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-03-11T00:36:58.359 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-03-11T00:36:58.490 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-03-11T00:36:58.494 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-03-11T00:36:58.664 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46 2021-03-11T00:36:58.666 INFO:teuthology.task.ansible.out:TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC 2021-03-11T00:36:58.666 INFO:teuthology.task.ansible.out:1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-03-11T00:36:58.808 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-03-11T00:36:58.820 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-03-11T00:36:58.822 INFO:teuthology.task.ansible.out: 2021-03-11T00:36:58.950 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-03-11T00:36:58.951 INFO:teuthology.task.ansible.out: 2021-03-11T00:36:58.952 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-03-11T00:36:59.093 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "pre 2021-03-11T00:36:59.094 INFO:teuthology.task.ansible.out:sent", "user": "xiaoxichen", "validate_certs": true} 2021-03-11T00:36:59.229 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-03-11T00:36:59.241 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-03-11T00:36:59.367 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-03-11T00:36:59.389 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-03-11T00:36:59.527 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQv 2021-03-11T00:36:59.529 INFO:teuthology.task.ansible.out:UxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@c 2021-03-11T00:36:59.531 INFO:teuthology.task.ansible.out:lusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQvUxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2021-03-11T00:36:59.666 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2021-03-11T00:36:59.669 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2021-03-11T00:36:59.794 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-03-11T00:36:59.796 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-03-11T00:36:59.922 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-03-11T00:36:59.934 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-03-11T00:37:00.071 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2021-03-11T00:37:00.072 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2021-03-11T00:37:00.212 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2021-03-11T00:37:00.214 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2021-03-11T00:37:00.342 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2021-03-11T00:37:00.343 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2021-03-11T00:37:00.480 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-03-11T00:37:00.481 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-03-11T00:37:00.609 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-03-11T00:37:00.611 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-03-11T00:37:00.748 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-03-11T00:37:00.750 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-03-11T00:37:00.876 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1 2021-03-11T00:37:00.878 INFO:teuthology.task.ansible.out:OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6 2021-03-11T00:37:00.879 INFO:teuthology.task.ansible.out:UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-03-11T00:37:01.006 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyK 2021-03-11T00:37:01.008 INFO:teuthology.task.ansible.out:z2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2021-03-11T00:37:01.028 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyK 2021-03-11T00:37:01.030 INFO:teuthology.task.ansible.out:z2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2021-03-11T00:37:01.157 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-03-11T00:37:01.159 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:01.159 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-03-11T00:37:01.288 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-03-11T00:37:01.289 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-03-11T00:37:01.429 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-03-11T00:37:01.441 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-03-11T00:37:01.569 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nu 2021-03-11T00:37:01.572 INFO:teuthology.task.ansible.out:ll, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-03-11T00:37:01.592 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nu 2021-03-11T00:37:01.594 INFO:teuthology.task.ansible.out:ll, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-03-11T00:37:01.717 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjE 2021-03-11T00:37:01.720 INFO:teuthology.task.ansible.out:fdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+d 2021-03-11T00:37:01.722 INFO:teuthology.task.ansible.out:TzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2021-03-11T00:37:01.740 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjE 2021-03-11T00:37:01.742 INFO:teuthology.task.ansible.out:fdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+d 2021-03-11T00:37:01.744 INFO:teuthology.task.ansible.out:TzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2021-03-11T00:37:01.851 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-03-11T00:37:01.890 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-03-11T00:37:01.997 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-03-11T00:37:02.030 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-03-11T00:37:02.158 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-03-11T00:37:02.185 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-03-11T00:37:02.315 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-03-11T00:37:02.337 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-03-11T00:37:02.450 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-03-11T00:37:02.483 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-03-11T00:37:02.609 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home 2021-03-11T00:37:02.611 INFO:teuthology.task.ansible.out:/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2021-03-11T00:37:02.627 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home 2021-03-11T00:37:02.629 INFO:teuthology.task.ansible.out:/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2021-03-11T00:37:02.759 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-03-11T00:37:02.761 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-03-11T00:37:02.918 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-03-11T00:37:02.919 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-03-11T00:37:03.051 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": " 2021-03-11T00:37:03.053 INFO:teuthology.task.ansible.out:gabrioux", "validate_certs": true} ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": 2021-03-11T00:37:03.054 INFO:teuthology.task.ansible.out: null, "state": "present", "user": "gabrioux", "validate_certs": true} 2021-03-11T00:37:03.179 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-03-11T00:37:03.181 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-03-11T00:37:03.318 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-03-11T00:37:03.319 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-03-11T00:37:03.450 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kk 2021-03-11T00:37:03.452 INFO:teuthology.task.ansible.out:ke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-03-11T00:37:03.462 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kk 2021-03-11T00:37:03.464 INFO:teuthology.task.ansible.out:ke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-03-11T00:37:03.579 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-03-11T00:37:03.591 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-03-11T00:37:03.719 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-03-11T00:37:03.730 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-03-11T00:37:03.849 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-03-11T00:37:03.872 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-03-11T00:37:03.980 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-03-11T00:37:04.013 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-03-11T00:37:04.129 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-03-11T00:37:04.162 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-03-11T00:37:04.264 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-03-11T00:37:04.296 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-03-11T00:37:04.405 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-03-11T00:37:04.459 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-03-11T00:37:04.564 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-03-11T00:37:04.607 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-03-11T00:37:04.694 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-03-11T00:37:04.748 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-03-11T00:37:04.830 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-03-11T00:37:04.884 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-03-11T00:37:04.969 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-03-11T00:37:05.033 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-03-11T00:37:05.118 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-03-11T00:37:05.171 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-03-11T00:37:05.267 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-03-11T00:37:05.327 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-03-11T00:37:05.440 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-03-11T00:37:05.473 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-03-11T00:37:05.592 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-03-11T00:37:05.625 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-03-11T00:37:05.756 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-03-11T00:37:05.768 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-03-11T00:37:05.910 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-03-11T00:37:05.921 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-03-11T00:37:06.050 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-03-11T00:37:06.062 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-03-11T00:37:06.198 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-03-11T00:37:06.202 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-03-11T00:37:06.342 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCM 2021-03-11T00:37:06.344 INFO:teuthology.task.ansible.out:bXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-03-11T00:37:06.488 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1 2021-03-11T00:37:06.490 INFO:teuthology.task.ansible.out:lXTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-03-11T00:37:06.499 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1 2021-03-11T00:37:06.501 INFO:teuthology.task.ansible.out:lXTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-03-11T00:37:06.632 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-03-11T00:37:06.646 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-03-11T00:37:06.790 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-03-11T00:37:06.812 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-03-11T00:37:06.990 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-03-11T00:37:06.993 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-03-11T00:37:07.134 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0 2021-03-11T00:37:07.136 INFO:teuthology.task.ansible.out:+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2021-03-11T00:37:07.255 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.locald 2021-03-11T00:37:07.256 INFO:teuthology.task.ansible.out:omain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2021-03-11T00:37:07.266 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.locald 2021-03-11T00:37:07.268 INFO:teuthology.task.ansible.out:omain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2021-03-11T00:37:07.403 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq 2021-03-11T00:37:07.404 INFO:teuthology.task.ansible.out:2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-03-11T00:37:07.414 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq 2021-03-11T00:37:07.416 INFO:teuthology.task.ansible.out:2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-03-11T00:37:07.540 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-03-11T00:37:07.542 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-03-11T00:37:07.668 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-03-11T00:37:07.669 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-03-11T00:37:07.801 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-03-11T00:37:07.811 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-03-11T00:37:07.949 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-03-11T00:37:07.952 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-03-11T00:37:08.110 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-03-11T00:37:08.111 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-03-11T00:37:08.269 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-03-11T00:37:08.271 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-03-11T00:37:08.401 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-03-11T00:37:08.414 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-03-11T00:37:08.552 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loc 2021-03-11T00:37:08.554 INFO:teuthology.task.ansible.out:al", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-03-11T00:37:08.564 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loc 2021-03-11T00:37:08.566 INFO:teuthology.task.ansible.out:al", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-03-11T00:37:08.708 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-03-11T00:37:08.719 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-03-11T00:37:08.884 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-03-11T00:37:08.886 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-03-11T00:37:09.048 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-03-11T00:37:09.070 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-03-11T00:37:09.197 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-03-11T00:37:09.230 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-03-11T00:37:09.336 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-03-11T00:37:09.363 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-03-11T00:37:09.469 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-03-11T00:37:09.492 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-03-11T00:37:09.611 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", 2021-03-11T00:37:09.613 INFO:teuthology.task.ansible.out:"manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-03-11T00:37:09.659 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", 2021-03-11T00:37:09.661 INFO:teuthology.task.ansible.out:"manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-03-11T00:37:09.744 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-03-11T00:37:09.787 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-03-11T00:37:09.882 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-03-11T00:37:09.946 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-03-11T00:37:10.032 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-03-11T00:37:10.138 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-03-11T00:37:10.192 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-03-11T00:37:10.277 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-03-11T00:37:10.333 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-03-11T00:37:10.431 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-03-11T00:37:10.485 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-03-11T00:37:10.559 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-03-11T00:37:10.639 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-03-11T00:37:10.705 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-03-11T00:37:10.819 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-03-11T00:37:10.820 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:10.853 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-03-11T00:37:11.006 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-03-11T00:37:11.085 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-03-11T00:37:11.186 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-03-11T00:37:11.252 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-03-11T00:37:11.405 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-03-11T00:37:11.425 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-03-11T00:37:11.558 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-03-11T00:37:11.608 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-03-11T00:37:11.759 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-03-11T00:37:11.770 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-03-11T00:37:11.928 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-03-11T00:37:11.940 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-03-11T00:37:12.101 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-03-11T00:37:12.102 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-03-11T00:37:12.262 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-03-11T00:37:12.264 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-03-11T00:37:12.404 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-03-11T00:37:12.415 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-03-11T00:37:12.544 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-03-11T00:37:12.567 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-03-11T00:37:12.674 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjL 2021-03-11T00:37:12.676 INFO:teuthology.task.ansible.out:d+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-03-11T00:37:12.707 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjL 2021-03-11T00:37:12.708 INFO:teuthology.task.ansible.out:d+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-03-11T00:37:12.816 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-03-11T00:37:12.849 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-03-11T00:37:12.947 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-03-11T00:37:12.980 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-03-11T00:37:13.099 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-03-11T00:37:13.110 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-03-11T00:37:13.238 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-03-11T00:37:13.250 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-03-11T00:37:13.403 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-03-11T00:37:13.405 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-03-11T00:37:13.521 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-03-11T00:37:13.532 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-03-11T00:37:13.684 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-03-11T00:37:13.695 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-03-11T00:37:13.824 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-03-11T00:37:13.856 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-03-11T00:37:13.964 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-03-11T00:37:13.996 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-03-11T00:37:14.084 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-03-11T00:37:14.118 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-03-11T00:37:14.213 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-03-11T00:37:14.246 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-03-11T00:37:14.361 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQKiUAlB8dxudAY60M7jnIJvs9yb2pNaT+BVv1Vk1kMs0XjPjcw9cyT3DtQ1xGLF367phRi/cfQbXOYbHG+uyBzQ4EEnBFvziktWRx4mfeWYxKjkjbfkVfpKPsxztqCbwm6+NYzMxGK2cExN7jUHnghHAjfA0IUpVVg9peBD3kUveLwS+UxuJAnuj+G4480LS6GDyLrhFEXU7uOIIlW4S9oLnU6vJL6GAzwyLE4ijTA8Ek03SJ01Ey/gpU9AqnQNp9u12l/7sJely6sI9pAuIzmS8Ht324hexxSWN5sFjXgLvLfFK83ziDBdrFBr0WXnWsFdRpHs2CwSq1JTkByqbN hmunjulu@hmunjulu.remote.csb", "key_options": null, "keyfile": "/home/hmunjulu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hmunjulu", "validate_certs": true} 2021-03-11T00:37:14.372 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQKiUAlB8dxudAY60M7jnIJvs9yb2pNaT+BVv1Vk1kMs0XjPjcw9cyT3DtQ1xGLF367phRi/cfQbXOYbHG+uyBzQ4EEnBFvziktWRx4mfeWYxKjkjbfkVfpKPsxztqCbwm6+NYzMxGK2cExN7jUHnghHAjfA0IUpVVg9peBD3kUveLwS+UxuJAnuj+G4480LS6GDyLrhFEXU7uOIIlW4S9oLnU6vJL6GAzwyLE4ijTA8Ek03SJ01Ey/gpU9AqnQNp9u12l/7sJely6sI9pAuIzmS8Ht324hexxSWN5sFjXgLvLfFK83ziDBdrFBr0WXnWsFdRpHs2CwSq1JTkByqbN hmunjulu@hmunjulu.remote.csb", "key_options": null, "keyfile": "/home/hmunjulu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hmunjulu", "validate_certs": true} 2021-03-11T00:37:14.482 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-03-11T00:37:14.504 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-03-11T00:37:14.614 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev", "key_options": null, "keyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-03-11T00:37:14.636 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev", "key_options": null, "keyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-03-11T00:37:14.735 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-03-11T00:37:14.768 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-03-11T00:37:14.864 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-03-11T00:37:14.900 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-03-11T00:37:15.007 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJ3ch+pU88/tX5TVVFl/1pPief8drYLRkK4BtfJk+xVNxX9qkDtOtiLAU+OugYF1/xk9hE9zTbm3UBL23unrzhlR80t8f2FiF1YFhHVzeZ/p/OunmkU0RFQzkMVuwKVOUNulw9FwpEZ/0Bjfxvaims0SBWaCzBgE5g0R+Ofdq/bVF82K8vU5Laa3KZ/cCdGzFgW00cBvPAKIpQ93g8TRqsCsFvR37nY9v5SU1hJ9araEp6LRA86tUt3EFtxD2tRcmqKGtCb0kOr+rRXWVGiaUBiedHxFudIy0+TNj4fNrJwpoC0hcBO9ek4Bh6hM7Zqzum9HpkPGlz84jkm+r+xMThaZCh8iFkgxyK+bw8B8smaVJD04ob9Wb4RYoFYfQ7TEkPkO9gr+UsSesbP9fWfucemUqwK0htOOtpc1YRwdOPcdPJWAfx9J5+YVL8BTWYBaTdmHslOKeUGM5fCob6QxC6Er9xweys9jh7MXttSpe013bY7P4ZDRF8uWvygpsytPE= amathuri@redhat.com", " 2021-03-11T00:37:15.009 INFO:teuthology.task.ansible.out:key_options": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} 2021-03-11T00:37:15.030 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJ3ch+pU88/tX5TVVFl/1pPief8drYLRkK4BtfJk+xVNxX9qkDtOtiLAU+OugYF1/xk9hE9zTbm3UBL23unrzhlR80t8f2FiF1YFhHVzeZ/p/OunmkU0RFQzkMVuwKVOUNulw9FwpEZ/0Bjfxvaims0SBWaCzBgE5g0R+Ofdq/bVF82K8vU5Laa3KZ/cCdGzFgW00cBvPAKIpQ93g8TRqsCsFvR37nY9v5SU1hJ9araEp6LRA86tUt3EFtxD2tRcmqKGtCb0kOr+rRXWVGiaUBiedHxFudIy0+TNj4fNrJwpoC0hcBO9ek4Bh6hM7Zqzum9HpkPGlz84jkm+r+xMThaZCh8iFkgxyK+bw8B8smaVJD04ob9Wb4RYoFYfQ7TEkPkO9gr+UsSesbP9fWfucemUqwK0htOOtpc1YRwdOPcdPJWAfx9J5+YVL8BTWYBaTdmHslOKeUGM5fCob6QxC6Er9xweys9jh7MXttSpe013bY7P4ZDRF8uWvygpsytPE= amathuri@redhat.com", " 2021-03-11T00:37:15.032 INFO:teuthology.task.ansible.out:key_options": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} 2021-03-11T00:37:15.147 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDr6H7hmtGCUrO3GppgMHwbmA4XT/Vipz/uLG/N8h6ZxtaLQKpCefRpdKcFlOdDwxkTmT5q9vmXQ8nl95lTGq+eeMxl4ho104ZD9e9AIYmgYqFy7/qNP8gkqduIPlypLL9m5rElTcGd7zqfMqgljMvAmnXLtZNJlIuFtszVU5WPI1QCwfQ3t5aHF6j66bpVoN9uWItVGB1LApQz9TtViWsQ9Hl8X4YauJToO+TkuSX+H/OGH7ZC6CPOzNZb3b3vF/jLMnTzXl86PJs1se4DjKkRGLVCVbr2WhLwQeZ8TEn3Hg1Gn5X8zNM5cTo6U8SnXBAe9ZYbuyfvghrNLdlZpmN+8l0/l5zC3z+N+sEzklXwHw4MQLF+KSloVFd9NN/kyenufTZAyp8VxllZ9TgdC+CPORIWyzmu1elOtYo7LNuE6CIh/Zm9B54SO6hVHhVoOCqe/dIyjeHJA0WKIZblK1XZEvWuu0E2T3jhxfe1GICyTukeB8WoBQH9DZ1yzMPt59E= nia@redhat.com", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} 2021-03-11T00:37:15.268 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDr6H7hmtGCUrO3GppgMHwbmA4XT/Vipz/uLG/N8h6ZxtaLQKpCefRpdKcFlOdDwxkTmT5q9vmXQ8nl95lTGq+eeMxl4ho104ZD9e9AIYmgYqFy7/qNP8gkqduIPlypLL9m5rElTcGd7zqfMqgljMvAmnXLtZNJlIuFtszVU5WPI1QCwfQ3t5aHF6j66bpVoN9uWItVGB1LApQz9TtViWsQ9Hl8X4YauJToO+TkuSX+H/OGH7ZC6CPOzNZb3b3vF/jLMnTzXl86PJs1se4DjKkRGLVCVbr2WhLwQeZ8TEn3Hg1Gn5X8zNM5cTo6U8SnXBAe9ZYbuyfvghrNLdlZpmN+8l0/l5zC3z+N+sEzklXwHw4MQLF+KSloVFd9NN/kyenufTZAyp8VxllZ9TgdC+CPORIWyzmu1elOtYo7LNuE6CIh/Zm9B54SO6hVHhVoOCqe/dIyjeHJA0WKIZblK1XZEvWuu0E2T3jhxfe1GICyTukeB8WoBQH9DZ1yzMPt59E= nia@redhat.com", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} 2021-03-11T00:37:15.387 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:15 +0000 (0:00:20.427) 0:00:52.005 ******** 2021-03-11T00:37:15.418 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:15 +0000 (0:00:00.031) 0:00:52.036 ******** 2021-03-11T00:37:15.599 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-03-11T00:37:15.601 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:15.611 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:15.623 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:15.625 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:15.637 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:15.639 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:15.650 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:15.651 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:15.656 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:15.665 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:15.667 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:15.678 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:15.681 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:15.692 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:15.695 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:15.705 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:15.869 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2021-03-11T00:37:15.871 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2021-03-11T00:37:15.883 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:15.885 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:15.897 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:15.898 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:16.019 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2021-03-11T00:37:16.043 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2021-03-11T00:37:16.183 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2021-03-11T00:37:16.185 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2021-03-11T00:37:16.195 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.197 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.207 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.208 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.220 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.231 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.233 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488f 2021-03-11T00:37:16.234 INFO:teuthology.task.ansible.out:f621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.243 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.245 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.256 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.258 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.269 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.270 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06 2021-03-11T00:37:16.272 INFO:teuthology.task.ansible.out:b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.282 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.284 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.295 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.297 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.308 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.309 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.320 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.322 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.351 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2 2021-03-11T00:37:16.353 INFO:teuthology.task.ansible.out:794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.356 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.367 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.369 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.380 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.381 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:16.383 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.393 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.405 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.412 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.413 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.414 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.415 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.426 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.437 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.439 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.450 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.461 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.462 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.473 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.475 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:16.487 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.488 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.490 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.501 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.504 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:16.506 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.506 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.508 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.508 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.520 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.522 INFO:teuthology.task.ansible.out: skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.535 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.537 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.547 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.549 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.558 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.561 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.571 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.583 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.585 INFO:teuthology.task.ansible.out: skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.588 INFO:teuthology.task.ansible.out: skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.590 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.594 INFO:teuthology.task.ansible.out: skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.603 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.605 INFO:teuthology.task.ansible.out: skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.617 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.628 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.639 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.652 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.663 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.675 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.689 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.701 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.704 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.719 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.732 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.743 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.745 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.756 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.768 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.780 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.782 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.793 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.795 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.805 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.806 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.819 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.820 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.837 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.847 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.849 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.860 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.863 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.873 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.885 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.887 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.897 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.899 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.911 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.913 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.927 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.936 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.938 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.949 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.951 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.961 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.963 INFO:teuthology.task.ansible.out: skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.974 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.975 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.985 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.987 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.998 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:16.999 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.000 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.011 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.013 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.025 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.027 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.038 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.039 INFO:teuthology.task.ansible.out: skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.040 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.042 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.042 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.053 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.060 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.065 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.067 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.078 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.080 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.088 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.090 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.105 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.107 INFO:teuthology.task.ansible.out: skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.108 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.108 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.121 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.132 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.135 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.146 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.148 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.161 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.170 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.181 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.183 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.203 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.205 INFO:teuthology.task.ansible.out: skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.216 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.217 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.218 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.230 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.230 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.232 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.243 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.245 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.255 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.266 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.267 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.269 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.279 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.291 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.293 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.304 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.306 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.318 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.318 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.330 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.332 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.342 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.343 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.345 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.357 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.358 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.359 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.371 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.372 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.384 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.386 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.387 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.388 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.399 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.410 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.412 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.422 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.424 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.444 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.447 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.457 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.459 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.512 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.514 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.539 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:17 +0000 (0:00:02.120) 0:00:54.157 ******** 2021-03-11T00:37:17.568 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.569 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.571 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.583 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.595 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.597 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.607 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.618 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.629 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.631 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.641 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.662 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.674 INFO:teuthology.task.ansible.out: TASK [Filter the revoked_users list] ******************************************* skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.676 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.704 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:17 +0000 (0:00:00.165) 0:00:54.322 ******** 2021-03-11T00:37:17.714 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.716 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.717 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.807 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:17 +0000 (0:00:00.102) 0:00:54.425 ******** 2021-03-11T00:37:17.927 INFO:teuthology.task.ansible.out: TASK [Filter the revoked_users list] ******************************************* 2021-03-11T00:37:17.929 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:17.954 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:17 +0000 (0:00:00.147) 0:00:54.572 ******** 2021-03-11T00:37:18.013 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-03-11T00:37:18.015 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2021-03-11T00:37:18.175 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-03-11T00:37:18.198 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2021-03-11T00:37:18.200 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:18.336 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2021-03-11T00:37:18.360 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-03-11T00:37:18.488 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-03-11T00:37:18.531 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2021-03-11T00:37:18.656 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2021-03-11T00:37:18.689 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-03-11T00:37:18.818 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2021-03-11T00:37:18.830 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2021-03-11T00:37:18.961 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-03-11T00:37:18.983 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2021-03-11T00:37:19.122 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-03-11T00:37:19.123 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-03-11T00:37:19.261 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-03-11T00:37:19.283 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-03-11T00:37:19.401 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-03-11T00:37:19.402 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:19.414 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-03-11T00:37:19.416 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:19.546 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-03-11T00:37:19.578 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-03-11T00:37:19.677 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-03-11T00:37:19.720 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-03-11T00:37:19.834 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-03-11T00:37:19.867 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-03-11T00:37:19.996 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-03-11T00:37:20.019 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-03-11T00:37:20.130 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-03-11T00:37:20.163 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-03-11T00:37:20.268 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-03-11T00:37:20.311 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-03-11T00:37:20.410 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-03-11T00:37:20.454 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-03-11T00:37:20.540 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-03-11T00:37:20.604 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-03-11T00:37:20.710 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-03-11T00:37:20.785 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-03-11T00:37:20.849 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-03-11T00:37:20.944 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-03-11T00:37:20.997 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-03-11T00:37:21.093 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-03-11T00:37:21.146 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2021-03-11T00:37:21.231 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-03-11T00:37:21.287 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-03-11T00:37:21.372 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2021-03-11T00:37:21.426 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-03-11T00:37:21.554 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-03-11T00:37:21.565 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-03-11T00:37:21.696 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-03-11T00:37:21.705 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-03-11T00:37:21.831 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-03-11T00:37:21.854 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-03-11T00:37:21.971 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-03-11T00:37:21.993 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-03-11T00:37:22.110 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-03-11T00:37:22.132 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-03-11T00:37:22.259 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-03-11T00:37:22.270 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-03-11T00:37:22.402 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-03-11T00:37:22.414 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-03-11T00:37:22.542 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-03-11T00:37:22.566 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-03-11T00:37:22.683 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-03-11T00:37:22.716 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-03-11T00:37:22.822 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-03-11T00:37:22.865 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-03-11T00:37:22.972 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-03-11T00:37:23.015 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-03-11T00:37:23.112 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-03-11T00:37:23.174 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-03-11T00:37:23.260 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-03-11T00:37:23.313 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-03-11T00:37:23.398 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-03-11T00:37:23.452 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-03-11T00:37:23.538 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-03-11T00:37:23.602 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": false, "item": "zyan", "name": "zyan", "state": "absent"} 2021-03-11T00:37:23.708 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-03-11T00:37:23.751 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-03-11T00:37:23.868 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": false, "item": "zyan", "name": "zyan", "state": "absent"} 2021-03-11T00:37:23.922 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-03-11T00:37:24.029 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-03-11T00:37:24.073 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-03-11T00:37:24.158 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-03-11T00:37:24.222 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-03-11T00:37:24.300 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-03-11T00:37:24.364 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-03-11T00:37:24.429 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-03-11T00:37:24.504 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-03-11T00:37:24.593 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-03-11T00:37:24.636 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-03-11T00:37:24.739 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-03-11T00:37:24.792 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-03-11T00:37:24.898 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-03-11T00:37:24.951 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-03-11T00:37:25.068 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-03-11T00:37:25.112 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-03-11T00:37:25.156 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:25 +0000 (0:00:07.201) 0:01:01.774 ******** 2021-03-11T00:37:25.216 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-03-11T00:37:25.375 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-03-11T00:37:25.432 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:25 +0000 (0:00:00.276) 0:01:02.050 ******** 2021-03-11T00:37:25.618 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2021-03-11T00:37:25.620 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => {"changed": true, "checksum": "1d9711ff879c231c67cb70d00ca0088797a377d0", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "80a55def4b505d08aff3687f81f74dc2", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615423045.2788138-25713-178649742095304/source", "state": "file", "uid": 0} 2021-03-11T00:37:25.646 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:25 +0000 (0:00:00.214) 0:01:02.264 ******** 2021-03-11T00:37:25.726 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2021-03-11T00:37:25.728 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => { "msg": "Host smithi124.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } 2021-03-11T00:37:25.752 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:25 +0000 (0:00:00.105) 0:01:02.370 ******** 2021-03-11T00:37:25.855 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** ok: [smithi124.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2021-03-11T00:37:25.884 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:25 +0000 (0:00:00.131) 0:01:02.502 ******** 2021-03-11T00:37:25.892 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2021-03-11T00:37:25.894 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => {"changed": true, "checksum": "1d9711ff879c231c67cb70d00ca0088797a377d0", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "80a55def4b505d08aff3687f81f74dc2", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615423045.5187612-25781-109055884146049/source", "state": "file", "uid": 0} 2021-03-11T00:37:25.919 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:25 +0000 (0:00:00.035) 0:01:02.537 ******** 2021-03-11T00:37:25.966 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:25.985 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:25 +0000 (0:00:00.066) 0:01:02.603 ******** 2021-03-11T00:37:26.007 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2021-03-11T00:37:26.008 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => { "msg": "Host smithi119.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } 2021-03-11T00:37:26.037 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:26 +0000 (0:00:00.050) 0:01:02.654 ******** 2021-03-11T00:37:26.064 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2021-03-11T00:37:26.066 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:26.089 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:26 +0000 (0:00:00.053) 0:01:02.708 ******** 2021-03-11T00:37:26.128 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2021-03-11T00:37:26.130 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2021-03-11T00:37:26.154 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:26 +0000 (0:00:00.064) 0:01:02.772 ******** 2021-03-11T00:37:26.170 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2021-03-11T00:37:26.172 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:26.204 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:26 +0000 (0:00:00.050) 0:01:02.822 ******** 2021-03-11T00:37:26.225 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2021-03-11T00:37:26.227 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:26.251 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:26 +0000 (0:00:00.046) 0:01:02.869 ******** 2021-03-11T00:37:26.279 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2021-03-11T00:37:26.281 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:26.304 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:26 +0000 (0:00:00.053) 0:01:02.922 ******** 2021-03-11T00:37:26.353 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2021-03-11T00:37:26.355 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:26.379 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:26 +0000 (0:00:00.074) 0:01:02.997 ******** 2021-03-11T00:37:26.385 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2021-03-11T00:37:26.387 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:26.426 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:26 +0000 (0:00:00.047) 0:01:03.044 ******** 2021-03-11T00:37:26.429 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:26.469 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:26.496 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:26 +0000 (0:00:00.070) 0:01:03.115 ******** 2021-03-11T00:37:26.544 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:26.580 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:26 +0000 (0:00:00.083) 0:01:03.198 ******** 2021-03-11T00:37:26.601 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:26.633 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:26 +0000 (0:00:00.053) 0:01:03.251 ******** 2021-03-11T00:37:26.676 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:26.706 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:26 +0000 (0:00:00.073) 0:01:03.325 ******** 2021-03-11T00:37:26.708 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:26.736 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2021-03-11T00:37:26.738 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:26.798 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:26 +0000 (0:00:00.091) 0:01:03.416 ******** 2021-03-11T00:37:26.805 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2021-03-11T00:37:26.807 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:26.831 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:26 +0000 (0:00:00.033) 0:01:03.450 ******** 2021-03-11T00:37:26.861 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:26.891 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:26 +0000 (0:00:00.059) 0:01:03.509 ******** 2021-03-11T00:37:26.934 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2021-03-11T00:37:27.088 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi124.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} Thursday 11 March 2021 00:37:26 +0000 (0:00:00.070) 0:01:03.579 ******** TASK [common : Install yum on Fedora 22 and later] ***************************** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Thursday 11 March 2021 00:37:27 +0000 (0:00:00.054) 0:01:03.634 ******** 2021-03-11T00:37:27.100 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2021-03-11T00:37:27.102 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:27.130 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:27 +0000 (0:00:00.113) 0:01:03.748 ******** 2021-03-11T00:37:27.139 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2021-03-11T00:37:27.141 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:27.175 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:27 +0000 (0:00:00.044) 0:01:03.793 ******** 2021-03-11T00:37:27.252 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2021-03-11T00:37:27.254 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi119.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-11T00:37:27.287 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:27 +0000 (0:00:00.112) 0:01:03.905 ******** 2021-03-11T00:37:27.404 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2021-03-11T00:37:27.406 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:27.431 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:27 +0000 (0:00:00.144) 0:01:04.049 ******** 2021-03-11T00:37:27.443 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2021-03-11T00:37:27.445 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:27.479 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:27 +0000 (0:00:00.047) 0:01:04.097 ******** 2021-03-11T00:37:27.520 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:27.521 INFO:teuthology.task.ansible.out:TASK [common : Determine if node is registered with subscription-manager.] ***** skipping: [smithi124.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-11T00:37:27.554 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:27 +0000 (0:00:00.074) 0:01:04.171 ******** 2021-03-11T00:37:27.592 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2021-03-11T00:37:27.594 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:27.624 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:27 +0000 (0:00:00.070) 0:01:04.242 ******** 2021-03-11T00:37:27.653 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:27.655 INFO:teuthology.task.ansible.out:TASK [common : Set rhsm_registered if we're already registered] **************** skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:27.681 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:27 +0000 (0:00:00.057) 0:01:04.299 ******** 2021-03-11T00:37:27.699 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2021-03-11T00:37:27.701 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-11T00:37:27.731 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:27 +0000 (0:00:00.049) 0:01:04.349 ******** 2021-03-11T00:37:27.770 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2021-03-11T00:37:27.772 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-11T00:37:27.800 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:27 +0000 (0:00:00.069) 0:01:04.419 ******** 2021-03-11T00:37:27.819 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2021-03-11T00:37:27.821 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:27.862 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:27 +0000 (0:00:00.061) 0:01:04.480 ******** 2021-03-11T00:37:27.881 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2021-03-11T00:37:27.883 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:27.908 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:27 +0000 (0:00:00.046) 0:01:04.526 ******** 2021-03-11T00:37:27.947 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2021-03-11T00:37:27.950 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-11T00:37:27.977 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:27 +0000 (0:00:00.068) 0:01:04.595 ******** 2021-03-11T00:37:28.026 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2021-03-11T00:37:28.028 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:28.050 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:28 +0000 (0:00:00.073) 0:01:04.668 ******** 2021-03-11T00:37:28.068 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2021-03-11T00:37:28.070 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:28.096 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:28 +0000 (0:00:00.045) 0:01:04.714 ******** 2021-03-11T00:37:28.148 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2021-03-11T00:37:28.150 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:28.176 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:28 +0000 (0:00:00.080) 0:01:04.795 ******** 2021-03-11T00:37:28.196 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2021-03-11T00:37:28.197 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:28.238 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:28 +0000 (0:00:00.060) 0:01:04.856 ******** 2021-03-11T00:37:28.287 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2021-03-11T00:37:28.288 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:28.316 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:28 +0000 (0:00:00.078) 0:01:04.935 ******** 2021-03-11T00:37:28.325 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:28.327 INFO:teuthology.task.ansible.out:TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:28.360 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:28 +0000 (0:00:00.043) 0:01:04.978 ******** 2021-03-11T00:37:28.400 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2021-03-11T00:37:28.402 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-11T00:37:28.439 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:28 +0000 (0:00:00.078) 0:01:05.057 ******** 2021-03-11T00:37:28.462 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:28.475 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:28 +0000 (0:00:00.036) 0:01:05.093 ******** 2021-03-11T00:37:28.510 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-03-11T00:37:28.512 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:28.555 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:28 +0000 (0:00:00.079) 0:01:05.173 ******** 2021-03-11T00:37:28.565 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2021-03-11T00:37:28.566 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-11T00:37:28.602 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:28 +0000 (0:00:00.046) 0:01:05.220 ******** 2021-03-11T00:37:28.637 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2021-03-11T00:37:28.639 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:28.678 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:28 +0000 (0:00:00.065) 0:01:05.286 ******** 2021-03-11T00:37:28.688 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-03-11T00:37:28.690 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:28.720 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:28 +0000 (0:00:00.052) 0:01:05.338 ******** 2021-03-11T00:37:28.751 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2021-03-11T00:37:28.752 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:28.778 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:28 +0000 (0:00:00.058) 0:01:05.396 ******** 2021-03-11T00:37:28.807 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2021-03-11T00:37:28.809 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:28.836 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:28 +0000 (0:00:00.057) 0:01:05.454 ******** 2021-03-11T00:37:28.876 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2021-03-11T00:37:28.878 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:28.904 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:28 +0000 (0:00:00.068) 0:01:05.523 ******** 2021-03-11T00:37:28.945 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2021-03-11T00:37:28.947 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:28.977 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:28 +0000 (0:00:00.071) 0:01:05.595 ******** 2021-03-11T00:37:29.016 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2021-03-11T00:37:29.018 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:29.047 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:29 +0000 (0:00:00.070) 0:01:05.665 ******** 2021-03-11T00:37:29.097 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2021-03-11T00:37:29.099 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:29.126 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:29 +0000 (0:00:00.078) 0:01:05.744 ******** 2021-03-11T00:37:29.143 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2021-03-11T00:37:29.145 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:29.172 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:29 +0000 (0:00:00.046) 0:01:05.790 ******** 2021-03-11T00:37:29.233 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2021-03-11T00:37:29.235 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:29.260 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:29 +0000 (0:00:00.088) 0:01:05.879 ******** 2021-03-11T00:37:29.269 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2021-03-11T00:37:29.270 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-11T00:37:29.299 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:29 +0000 (0:00:00.038) 0:01:05.917 ******** 2021-03-11T00:37:29.352 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:29.379 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:29 +0000 (0:00:00.079) 0:01:05.997 ******** 2021-03-11T00:37:29.382 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:29.399 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2021-03-11T00:37:29.400 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:29.429 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:29 +0000 (0:00:00.049) 0:01:06.047 ******** 2021-03-11T00:37:29.458 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2021-03-11T00:37:29.460 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-11T00:37:29.490 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:29 +0000 (0:00:00.057) 0:01:06.105 ******** 2021-03-11T00:37:29.507 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2021-03-11T00:37:29.509 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:29.539 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:29 +0000 (0:00:00.052) 0:01:06.158 ******** 2021-03-11T00:37:29.593 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:29.620 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:29 +0000 (0:00:00.080) 0:01:06.238 ******** 2021-03-11T00:37:29.641 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2021-03-11T00:37:29.643 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:29.668 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:29 +0000 (0:00:00.047) 0:01:06.286 ******** 2021-03-11T00:37:29.707 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:29.732 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:29 +0000 (0:00:00.064) 0:01:06.351 ******** 2021-03-11T00:37:29.761 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2021-03-11T00:37:29.762 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:29.773 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:29.800 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:29 +0000 (0:00:00.067) 0:01:06.418 ******** 2021-03-11T00:37:29.827 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2021-03-11T00:37:29.829 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:29.856 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:29 +0000 (0:00:00.055) 0:01:06.474 ******** 2021-03-11T00:37:29.884 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-03-11T00:37:29.885 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2021-03-11T00:37:29.910 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:29 +0000 (0:00:00.054) 0:01:06.528 ******** 2021-03-11T00:37:29.959 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2021-03-11T00:37:29.961 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:29.995 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:29 +0000 (0:00:00.085) 0:01:06.614 ******** 2021-03-11T00:37:30.002 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-03-11T00:37:30.004 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:30.029 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:30 +0000 (0:00:00.033) 0:01:06.647 ******** 2021-03-11T00:37:30.088 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-03-11T00:37:30.090 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2021-03-11T00:37:30.116 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:30 +0000 (0:00:00.086) 0:01:06.734 ******** 2021-03-11T00:37:30.186 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-03-11T00:37:30.188 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:30.214 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:30 +0000 (0:00:00.098) 0:01:06.832 ******** 2021-03-11T00:37:30.331 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-03-11T00:37:30.333 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.073142", "end": "2021-03-11 00:37:30.566852", "rc": 0, "start": "2021-03-11 00:37:30.493710", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2021-03-11T00:37:30.356 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:30 +0000 (0:00:00.142) 0:01:06.975 ******** 2021-03-11T00:37:30.442 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.012843", "end": "2021-03-11 00:37:30.318977", "rc": 0, "start": "2021-03-11 00:37:30.306134", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2021-03-11T00:37:30.475 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:30 +0000 (0:00:00.118) 0:01:07.093 ******** 2021-03-11T00:37:30.482 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2021-03-11T00:37:30.483 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:30.508 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:30 +0000 (0:00:00.032) 0:01:07.126 ******** 2021-03-11T00:37:30.550 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:30.576 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:30 +0000 (0:00:00.068) 0:01:07.195 ******** 2021-03-11T00:37:30.581 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:30.592 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2021-03-11T00:37:30.594 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:30.621 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:30 +0000 (0:00:00.044) 0:01:07.239 ******** 2021-03-11T00:37:30.670 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:30.699 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:30 +0000 (0:00:00.078) 0:01:07.318 ******** 2021-03-11T00:37:30.734 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** 2021-03-11T00:37:30.736 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "skip_reason": "Conditional result was False"} 2021-03-11T00:37:30.738 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "skip_reason": "Conditional result was False"} 2021-03-11T00:37:30.739 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "skip_reason": "Conditional result was False"} 2021-03-11T00:37:30.790 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:30 +0000 (0:00:00.090) 0:01:07.408 ******** 2021-03-11T00:37:30.808 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "skip_reason": "Conditional result was False"} 2021-03-11T00:37:30.810 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "skip_reason": "Conditional result was False"} 2021-03-11T00:37:30.819 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "skip_reason": "Conditional result was False"} 2021-03-11T00:37:30.831 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "skip_reason": "Conditional result was False"} 2021-03-11T00:37:30.856 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:30 +0000 (0:00:00.066) 0:01:07.475 ******** 2021-03-11T00:37:30.863 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-03-11T00:37:30.865 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:30.890 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:30 +0000 (0:00:00.033) 0:01:07.508 ******** 2021-03-11T00:37:30.929 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:30.965 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:30 +0000 (0:00:00.075) 0:01:07.583 ******** 2021-03-11T00:37:30.979 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2021-03-11T00:37:30.981 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:31.008 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:31 +0000 (0:00:00.042) 0:01:07.626 ******** 2021-03-11T00:37:31.081 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:31.115 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:31 +0000 (0:00:00.107) 0:01:07.733 ******** 2021-03-11T00:37:31.145 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:31.171 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:31 +0000 (0:00:00.055) 0:01:07.789 ******** 2021-03-11T00:37:31.191 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:31.223 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:31 +0000 (0:00:00.052) 0:01:07.841 ******** 2021-03-11T00:37:31.284 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2021-03-11T00:37:31.286 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:31.308 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:31 +0000 (0:00:00.084) 0:01:07.926 ******** 2021-03-11T00:37:31.316 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:31.318 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:31.344 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:31 +0000 (0:00:00.035) 0:01:07.962 ******** 2021-03-11T00:37:31.407 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2021-03-11T00:37:31.410 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:31.446 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:31 +0000 (0:00:00.102) 0:01:08.064 ******** 2021-03-11T00:37:31.453 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:31.485 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:31 +0000 (0:00:00.038) 0:01:08.103 ******** 2021-03-11T00:37:32.264 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:32.266 INFO:teuthology.task.ansible.out:TASK [common : Set the hardware clock] ***************************************** changed: [smithi124.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.658316", "end": "2021-03-11 00:37:32.500443", "rc": 0, "start": "2021-03-11 00:37:31.842127", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-11T00:37:32.303 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:32 +0000 (0:00:00.818) 0:01:08.921 ******** 2021-03-11T00:37:32.378 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2021-03-11T00:37:32.379 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:32.401 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:32 +0000 (0:00:00.098) 0:01:09.019 ******** 2021-03-11T00:37:32.620 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2021-03-11T00:37:32.622 INFO:teuthology.task.ansible.out:[WARNING]: Consider using the apt module rather than running 'apt-get'. If you need to use command because apt is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi124.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.022141", "end": "2021-03-11 00:37:32.857386", "rc": 0, "start": "2021-03-11 00:37:32.835245", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-11T00:37:32.661 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:32 +0000 (0:00:00.258) 0:01:09.278 ******** 2021-03-11T00:37:32.678 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2021-03-11T00:37:32.679 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.999221", "end": "2021-03-11 00:37:32.500300", "rc": 0, "start": "2021-03-11 00:37:31.501079", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-11T00:37:32.719 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:32 +0000 (0:00:00.059) 0:01:09.337 ******** 2021-03-11T00:37:32.817 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:32.842 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:32 +0000 (0:00:00.122) 0:01:09.460 ******** 2021-03-11T00:37:33.099 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2021-03-11T00:37:33.100 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.024373", "end": "2021-03-11 00:37:32.959517", "rc": 0, "start": "2021-03-11 00:37:32.935144", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-11T00:37:33.127 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:33 +0000 (0:00:00.285) 0:01:09.745 ******** 2021-03-11T00:37:36.233 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2021-03-11T00:37:36.234 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1615422969, "cache_updated": false, "changed": false} 2021-03-11T00:37:36.256 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:36 +0000 (0:00:03.128) 0:01:12.874 ******** 2021-03-11T00:37:36.407 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1615422974, "cache_updated": false, "changed": false} 2021-03-11T00:37:36.438 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:36 +0000 (0:00:00.181) 0:01:13.056 ******** 2021-03-11T00:37:37.193 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2021-03-11T00:37:37.195 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"cache_update_time": 1615422969, "cache_updated": false, "changed": false} 2021-03-11T00:37:37.218 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:37 +0000 (0:00:00.780) 0:01:13.836 ******** 2021-03-11T00:37:37.305 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2021-03-11T00:37:37.307 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:37.331 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:37 +0000 (0:00:00.112) 0:01:13.949 ******** 2021-03-11T00:37:37.392 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2021-03-11T00:37:37.394 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"cache_update_time": 1615422974, "cache_updated": false, "changed": false} 2021-03-11T00:37:37.426 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:37 +0000 (0:00:00.095) 0:01:14.045 ******** 2021-03-11T00:37:37.517 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2021-03-11T00:37:37.519 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:37.541 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:37 +0000 (0:00:00.114) 0:01:14.159 ******** 2021-03-11T00:37:38.077 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2021-03-11T00:37:38.103 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:38 +0000 (0:00:00.561) 0:01:14.721 ******** 2021-03-11T00:37:38.114 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2021-03-11T00:37:38.143 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:38 +0000 (0:00:00.039) 0:01:14.761 ******** 2021-03-11T00:37:39.036 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2021-03-11T00:37:39.038 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"cache_update_time": 1615422969, "cache_updated": false, "changed": false} 2021-03-11T00:37:39.061 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:39 +0000 (0:00:00.917) 0:01:15.679 ******** 2021-03-11T00:37:39.089 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"cache_update_time": 1615422974, "cache_updated": false, "changed": false} 2021-03-11T00:37:39.114 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:39 +0000 (0:00:00.053) 0:01:15.732 ******** 2021-03-11T00:37:39.488 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-03-11T00:37:39.490 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2021-03-11T00:37:39.617 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2021-03-11T00:37:39.802 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2021-03-11T00:37:39.932 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2021-03-11T00:37:40.104 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2021-03-11T00:37:40.131 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:40 +0000 (0:00:01.016) 0:01:16.749 ******** 2021-03-11T00:37:40.224 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2021-03-11T00:37:40.255 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:40 +0000 (0:00:00.123) 0:01:16.873 ******** 2021-03-11T00:37:40.274 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2021-03-11T00:37:40.276 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2021-03-11T00:37:40.305 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:40 +0000 (0:00:00.048) 0:01:16.922 ******** 2021-03-11T00:37:40.387 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2021-03-11T00:37:40.413 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:40 +0000 (0:00:00.108) 0:01:17.031 ******** 2021-03-11T00:37:40.717 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-03-11T00:37:40.719 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 13355, "state": "file", "uid": 0} 2021-03-11T00:37:40.803 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 13355, "state": "file", "uid": 0} 2021-03-11T00:37:41.035 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2021-03-11T00:37:41.088 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2021-03-11T00:37:41.344 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2021-03-11T00:37:41.371 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:41 +0000 (0:00:00.957) 0:01:17.989 ******** 2021-03-11T00:37:41.399 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2021-03-11T00:37:41.426 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:41 +0000 (0:00:00.055) 0:01:18.045 ******** 2021-03-11T00:37:41.594 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** ok: [smithi119.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2021-03-11T00:37:41.623 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:41 +0000 (0:00:00.197) 0:01:18.242 ******** 2021-03-11T00:37:41.681 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2021-03-11T00:37:41.706 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:41 +0000 (0:00:00.082) 0:01:18.324 ******** 2021-03-11T00:37:41.734 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2021-03-11T00:37:41.736 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:41.763 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:41 +0000 (0:00:00.056) 0:01:18.381 ******** 2021-03-11T00:37:41.823 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:41.856 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:41 +0000 (0:00:00.092) 0:01:18.474 ******** 2021-03-11T00:37:41.878 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2021-03-11T00:37:41.880 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-11T00:37:41.925 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:41 +0000 (0:00:00.068) 0:01:18.543 ******** 2021-03-11T00:37:41.968 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-11T00:37:42.029 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:42 +0000 (0:00:00.104) 0:01:18.647 ******** 2021-03-11T00:37:42.031 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:42.064 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2021-03-11T00:37:42.066 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-03-11T00:37:42.076 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-03-11T00:37:42.107 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:42 +0000 (0:00:00.078) 0:01:18.726 ******** 2021-03-11T00:37:42.180 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-03-11T00:37:42.192 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-03-11T00:37:42.203 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-03-11T00:37:42.205 INFO:teuthology.task.ansible.out: skipping: [smithi124.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-03-11T00:37:42.231 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:42 +0000 (0:00:00.123) 0:01:18.850 ******** 2021-03-11T00:37:42.246 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2021-03-11T00:37:42.248 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:42.277 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:42 +0000 (0:00:00.045) 0:01:18.895 ******** 2021-03-11T00:37:42.354 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:42.377 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:42 +0000 (0:00:00.100) 0:01:18.995 ******** 2021-03-11T00:37:42.381 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:43.491 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2021-03-11T00:37:43.493 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"cache_update_time": 1615422969, "cache_updated": false, "changed": false} 2021-03-11T00:37:43.528 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:43 +0000 (0:00:01.151) 0:01:20.147 ******** 2021-03-11T00:37:43.530 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:43.546 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"cache_update_time": 1615422974, "cache_updated": false, "changed": false} 2021-03-11T00:37:43.583 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:43 +0000 (0:00:00.054) 0:01:20.201 ******** 2021-03-11T00:37:44.275 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2021-03-11T00:37:44.299 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:44 +0000 (0:00:00.716) 0:01:20.917 ******** 2021-03-11T00:37:44.308 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2021-03-11T00:37:44.336 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:44 +0000 (0:00:00.036) 0:01:20.954 ******** 2021-03-11T00:37:44.782 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2021-03-11T00:37:44.788 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2021-03-11T00:37:44.812 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:44 +0000 (0:00:00.476) 0:01:21.431 ******** 2021-03-11T00:37:44.835 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2021-03-11T00:37:44.863 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:44 +0000 (0:00:00.050) 0:01:21.481 ******** 2021-03-11T00:37:44.865 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:45.013 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-03-11T00:37:45.015 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2021-03-11T00:37:45.089 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2021-03-11T00:37:45.136 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2021-03-11T00:37:45.159 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:45 +0000 (0:00:00.296) 0:01:21.777 ******** 2021-03-11T00:37:45.200 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2021-03-11T00:37:45.253 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:45 +0000 (0:00:00.092) 0:01:21.870 ******** 2021-03-11T00:37:45.271 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2021-03-11T00:37:45.273 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:45.309 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:45 +0000 (0:00:00.057) 0:01:21.927 ******** 2021-03-11T00:37:45.360 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:45.386 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:45 +0000 (0:00:00.076) 0:01:22.004 ******** 2021-03-11T00:37:45.414 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2021-03-11T00:37:45.415 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-11T00:37:45.443 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:45 +0000 (0:00:00.057) 0:01:22.061 ******** 2021-03-11T00:37:45.473 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-11T00:37:45.500 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:45 +0000 (0:00:00.057) 0:01:22.119 ******** 2021-03-11T00:37:45.561 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2021-03-11T00:37:45.563 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:45.586 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:45 +0000 (0:00:00.086) 0:01:22.205 ******** 2021-03-11T00:37:45.626 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:45.657 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:45 +0000 (0:00:00.070) 0:01:22.276 ******** 2021-03-11T00:37:45.660 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:46.089 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2021-03-11T00:37:46.090 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2021-03-11T00:37:46.117 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:46 +0000 (0:00:00.459) 0:01:22.735 ******** 2021-03-11T00:37:46.125 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2021-03-11T00:37:46.154 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:46 +0000 (0:00:00.036) 0:01:22.772 ******** 2021-03-11T00:37:46.880 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Thu 2021-03-11 00:35:19 UTC", "ActiveEnterTimestampMonotonic": "17169701", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "time-sync.target systemd-journald.socket remote-fs.target basic.target nss-lookup.target -.mount sysinit.target network.target systemd-tmpfiles-setup.service system.slice local-fs.target var-run.mount", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Thu 2021-03-11 00:35:19 UTC", "AssertTimestampMonotonic": "17168307", "Before": "multi-user.target xdm.service getty@tty1.service plymouth-quit.service shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Thu 2021-03-11 00:35:19 UTC", "ConditionTimestampMonotonic": "17168304", "ConfigurationDirectoryMode": "0755", "Confl 2021-03-11T00:37:46.882 INFO:teuthology.task.ansible.out:icts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nagios-nrpe-server.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1220", "ExecMainStartTimestamp": "Thu 2021-03-11 00:35:19 UTC", "ExecMainStartTimestampMonotonic": "17169662", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Thu 2021-03-11 00:35:19 UTC", "InactiveExitTimestampMonotonic": "17169701", "InvocationID": "22744412c1044dae94d6c9ca4f115d5f", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "Kill 2021-03-11T00:37:46.884 INFO:teuthology.task.ansible.out:Mode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1220", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount sysinit.target system.slice", "RequiresMountsFor": "/tmp 2021-03-11T00:37:46.886 INFO:teuthology.task.ansible.out:/var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Thu 2021-03-11 00:35:19 UTC", "StateChangeTimestampMonotonic": "17169701", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Thu 2021-03-11 00:35:19 UTC", "WatchdogTimestampMonotonic": "17169699", "WatchdogUSec": "0"}} 2021-03-11T00:37:46.913 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:46 +0000 (0:00:00.759) 0:01:23.531 ******** 2021-03-11T00:37:46.925 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Thu 2021-03-11 00:35:29 UTC", "ActiveEnterTimestampMonotonic": "17709036", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "remote-fs.target sysinit.target systemd-journald.socket var-run.mount time-sync.target basic.target -.mount network.target nss-lookup.target systemd-tmpfiles-setup.service local-fs.target system.slice", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Thu 2021-03-11 00:35:29 UTC", "AssertTimestampMonotonic": "17707823", "Before": "shutdown.target multi-user.target getty@tty1.service xdm.service plymouth-quit.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Thu 2021-03-11 00:35:29 UTC", "ConditionTimestampMonotonic": "17707823", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nagios-nrpe-ser 2021-03-11T00:37:46.927 INFO:teuthology.task.ansible.out:ver.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1210", "ExecMainStartTimestamp": "Thu 2021-03-11 00:35:29 UTC", "ExecMainStartTimestampMonotonic": "17708998", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Thu 2021-03-11 00:35:29 UTC", "InactiveExitTimestampMonotonic": "17709036", "InvocationID": "aeca6cbd7fe4484180043fd5aede198e", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "i 2021-03-11T00:37:46.929 INFO:teuthology.task.ansible.out:nfinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128028", "LimitNPROCSoft": "128028", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128028", "LimitSIGPENDINGSoft": "128028", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1210", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no" 2021-03-11T00:37:46.930 INFO:teuthology.task.ansible.out:, "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Thu 2021-03-11 00:35:29 UTC", "StateChangeTimestampMonotonic": "17709036", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Thu 2021-03-11 00:35:29 UTC", "WatchdogTimestampMonotonic": "17709035", "WatchdogUSec": "0"}} 2021-03-11T00:37:46.961 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:46 +0000 (0:00:00.047) 0:01:23.579 ******** 2021-03-11T00:37:47.011 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2021-03-11T00:37:47.013 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:47.038 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:47 +0000 (0:00:00.076) 0:01:23.656 ******** 2021-03-11T00:37:47.045 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:47.077 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:47 +0000 (0:00:00.039) 0:01:23.696 ******** 2021-03-11T00:37:47.122 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2021-03-11T00:37:47.124 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:47.149 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:47 +0000 (0:00:00.071) 0:01:23.767 ******** 2021-03-11T00:37:47.169 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:47.196 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:47 +0000 (0:00:00.046) 0:01:23.814 ******** 2021-03-11T00:37:47.247 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2021-03-11T00:37:47.249 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:47.280 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:47 +0000 (0:00:00.083) 0:01:23.898 ******** 2021-03-11T00:37:47.311 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:47.340 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:47 +0000 (0:00:00.060) 0:01:23.958 ******** 2021-03-11T00:37:47.371 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2021-03-11T00:37:47.373 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:47.405 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:47 +0000 (0:00:00.065) 0:01:24.024 ******** 2021-03-11T00:37:47.438 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:47.475 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:47 +0000 (0:00:00.069) 0:01:24.093 ******** 2021-03-11T00:37:47.503 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2021-03-11T00:37:47.505 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:47.531 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:47 +0000 (0:00:00.055) 0:01:24.149 ******** 2021-03-11T00:37:47.560 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:47.588 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:47 +0000 (0:00:00.057) 0:01:24.206 ******** 2021-03-11T00:37:47.616 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2021-03-11T00:37:47.623 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:47.646 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:47 +0000 (0:00:00.058) 0:01:24.264 ******** 2021-03-11T00:37:47.696 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:47.738 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:47 +0000 (0:00:00.079) 0:01:24.344 ******** 2021-03-11T00:37:47.750 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:47.757 INFO:teuthology.task.ansible.out:TASK [common : nrpe - Compile SELinux module file] ***************************** 2021-03-11T00:37:47.759 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:47.803 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:47 +0000 (0:00:00.077) 0:01:24.421 ******** 2021-03-11T00:37:47.844 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:47.874 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:47 +0000 (0:00:00.070) 0:01:24.492 ******** 2021-03-11T00:37:47.916 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2021-03-11T00:37:47.918 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:47.943 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:47 +0000 (0:00:00.068) 0:01:24.561 ******** 2021-03-11T00:37:47.972 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:47.999 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:47 +0000 (0:00:00.056) 0:01:24.618 ******** 2021-03-11T00:37:48.052 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2021-03-11T00:37:48.054 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:48.090 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:48 +0000 (0:00:00.087) 0:01:24.706 ******** 2021-03-11T00:37:48.158 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:48.196 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:48 +0000 (0:00:00.108) 0:01:24.814 ******** 2021-03-11T00:37:48.222 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:48.256 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:48 +0000 (0:00:00.059) 0:01:24.874 ******** 2021-03-11T00:37:48.360 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:48.400 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:48 +0000 (0:00:00.144) 0:01:25.018 ******** 2021-03-11T00:37:48.419 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2021-03-11T00:37:48.421 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2021-03-11T00:37:48.450 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:48 +0000 (0:00:00.050) 0:01:25.069 ******** 2021-03-11T00:37:48.454 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:48.619 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2021-03-11T00:37:48.651 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:48 +0000 (0:00:00.199) 0:01:25.268 ******** 2021-03-11T00:37:48.681 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2021-03-11T00:37:48.683 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "no_recommended_packages": ["collectl"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2021-03-11T00:37:48.734 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:48 +0000 (0:00:00.083) 0:01:25.352 ******** 2021-03-11T00:37:48.828 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "no_recommended_packages": ["collectl"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2021-03-11T00:37:48.870 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:48 +0000 (0:00:00.135) 0:01:25.488 ******** 2021-03-11T00:37:48.968 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:48.970 INFO:teuthology.task.ansible.out:TASK [testnode : Including major version specific variables.] ****************** 2021-03-11T00:37:48.972 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2021-03-11T00:37:49.003 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:49 +0000 (0:00:00.132) 0:01:25.621 ******** 2021-03-11T00:37:49.012 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2021-03-11T00:37:49.059 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:49 +0000 (0:00:00.056) 0:01:25.677 ******** 2021-03-11T00:37:49.061 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:49.114 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:49.122 INFO:teuthology.task.ansible.out:TASK [testnode : Including version specific variables.] ************************ 2021-03-11T00:37:49.124 INFO:teuthology.task.ansible.out: ok: [smithi119.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-03-11T00:37:49.151 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:49 +0000 (0:00:00.092) 0:01:25.770 ******** 2021-03-11T00:37:49.153 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:49.173 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-03-11T00:37:49.176 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:49.209 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:49 +0000 (0:00:00.057) 0:01:25.827 ******** 2021-03-11T00:37:49.338 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2021-03-11T00:37:49.340 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-03-11T00:37:49.364 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:49 +0000 (0:00:00.154) 0:01:25.982 ******** 2021-03-11T00:37:49.433 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-03-11T00:37:49.462 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:49 +0000 (0:00:00.094) 0:01:26.077 ******** 2021-03-11T00:37:49.585 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "name": "ubuntu", "state": "present", "system": false} 2021-03-11T00:37:49.612 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:49 +0000 (0:00:00.152) 0:01:26.230 ******** 2021-03-11T00:37:49.697 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "name": "ubuntu", "state": "present", "system": false} 2021-03-11T00:37:49.728 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:49 +0000 (0:00:00.116) 0:01:26.346 ******** 2021-03-11T00:37:49.752 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2021-03-11T00:37:49.753 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:49.795 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:49 +0000 (0:00:00.066) 0:01:26.413 ******** 2021-03-11T00:37:49.864 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:49.897 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:49 +0000 (0:00:00.102) 0:01:26.515 ******** 2021-03-11T00:37:49.992 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:50.017 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:50 +0000 (0:00:00.119) 0:01:26.635 ******** 2021-03-11T00:37:50.024 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:50.062 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:50 +0000 (0:00:00.045) 0:01:26.680 ******** 2021-03-11T00:37:50.268 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2021-03-11T00:37:50.269 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2021-03-11T00:37:50.300 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:50 +0000 (0:00:00.237) 0:01:26.918 ******** 2021-03-11T00:37:50.322 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2021-03-11T00:37:50.353 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:50 +0000 (0:00:00.053) 0:01:26.971 ******** 2021-03-11T00:37:50.869 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2021-03-11T00:37:50.899 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:50 +0000 (0:00:00.546) 0:01:27.518 ******** 2021-03-11T00:37:50.931 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2021-03-11T00:37:50.963 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:50 +0000 (0:00:00.063) 0:01:27.581 ******** 2021-03-11T00:37:51.007 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2021-03-11T00:37:51.009 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi119"}, "changed": false} 2021-03-11T00:37:51.047 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:51 +0000 (0:00:00.084) 0:01:27.665 ******** 2021-03-11T00:37:51.067 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi124"}, "changed": false} 2021-03-11T00:37:51.094 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:51 +0000 (0:00:00.045) 0:01:27.711 ******** 2021-03-11T00:37:52.142 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2021-03-11T00:37:52.144 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi119", "ansible_hostname": "smithi119", "ansible_nodename": "smithi119"}, "changed": false, "name": "smithi119"} 2021-03-11T00:37:52.174 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:52 +0000 (0:00:01.081) 0:01:28.792 ******** 2021-03-11T00:37:52.188 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi124", "ansible_hostname": "smithi124", "ansible_nodename": "smithi124"}, "changed": false, "name": "smithi124"} 2021-03-11T00:37:52.229 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:52 +0000 (0:00:00.054) 0:01:28.847 ******** 2021-03-11T00:37:52.768 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2021-03-11T00:37:52.815 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:52 +0000 (0:00:00.584) 0:01:29.432 ******** 2021-03-11T00:37:52.866 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2021-03-11T00:37:52.907 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:52 +0000 (0:00:00.093) 0:01:29.526 ******** 2021-03-11T00:37:53.716 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2021-03-11T00:37:53.739 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:53 +0000 (0:00:00.831) 0:01:30.357 ******** 2021-03-11T00:37:53.885 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2021-03-11T00:37:53.925 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:53 +0000 (0:00:00.186) 0:01:30.543 ******** 2021-03-11T00:37:54.436 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2021-03-11T00:37:54.438 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-03-11T00:37:54.461 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:54 +0000 (0:00:00.535) 0:01:31.079 ******** 2021-03-11T00:37:54.481 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-03-11T00:37:54.507 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:54 +0000 (0:00:00.046) 0:01:31.126 ******** 2021-03-11T00:37:54.589 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2021-03-11T00:37:54.591 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:54.614 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:54 +0000 (0:00:00.106) 0:01:31.232 ******** 2021-03-11T00:37:54.656 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:54.684 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:54 +0000 (0:00:00.069) 0:01:31.302 ******** 2021-03-11T00:37:54.731 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-03-11T00:37:54.732 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:54.760 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:54 +0000 (0:00:00.076) 0:01:31.378 ******** 2021-03-11T00:37:54.789 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:54.828 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:54 +0000 (0:00:00.067) 0:01:31.446 ******** 2021-03-11T00:37:54.877 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-03-11T00:37:54.881 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:54.908 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:54 +0000 (0:00:00.080) 0:01:31.527 ******** 2021-03-11T00:37:54.912 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:54.996 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:55.021 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:55 +0000 (0:00:00.112) 0:01:31.639 ******** 2021-03-11T00:37:55.023 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:55.061 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:55.087 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:55 +0000 (0:00:00.065) 0:01:31.705 ******** 2021-03-11T00:37:55.132 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:55.157 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:55 +0000 (0:00:00.070) 0:01:31.776 ******** 2021-03-11T00:37:55.159 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:55.222 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:55.249 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:55 +0000 (0:00:00.091) 0:01:31.867 ******** 2021-03-11T00:37:55.334 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:55.359 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:55 +0000 (0:00:00.109) 0:01:31.977 ******** 2021-03-11T00:37:55.381 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-03-11T00:37:55.382 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:55.408 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:55 +0000 (0:00:00.048) 0:01:32.026 ******** 2021-03-11T00:37:55.499 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:55.501 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:55.526 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:55 +0000 (0:00:00.115) 0:01:32.142 ******** 2021-03-11T00:37:55.565 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:55 +0000 (0:00:00.039) 0:01:32.181 ******** 2021-03-11T00:37:55.644 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2021-03-11T00:37:55.645 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:55.680 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:55 +0000 (0:00:00.117) 0:01:32.298 ******** 2021-03-11T00:37:55.705 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-03-11T00:37:55.707 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:55.732 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:55 +0000 (0:00:00.051) 0:01:32.350 ******** 2021-03-11T00:37:55.811 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-03-11T00:37:55.855 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:55 +0000 (0:00:00.123) 0:01:32.473 ******** 2021-03-11T00:37:55.864 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2021-03-11T00:37:55.866 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:55.896 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:55 +0000 (0:00:00.041) 0:01:32.514 ******** 2021-03-11T00:37:55.906 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:56.027 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-03-11T00:37:56.061 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:56 +0000 (0:00:00.164) 0:01:32.679 ******** 2021-03-11T00:37:56.108 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:56 +0000 (0:00:00.047) 0:01:32.726 ******** 2021-03-11T00:37:56.181 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2021-03-11T00:37:56.241 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:56 +0000 (0:00:00.132) 0:01:32.859 ******** 2021-03-11T00:37:56.250 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-03-11T00:37:56.280 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:56 +0000 (0:00:00.038) 0:01:32.898 ******** 2021-03-11T00:37:56.351 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:56.381 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:56 +0000 (0:00:00.098) 0:01:32.996 ******** 2021-03-11T00:37:56.408 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2021-03-11T00:37:56.436 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:56 +0000 (0:00:00.058) 0:01:33.054 ******** 2021-03-11T00:37:56.530 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* skipping: [smithi119.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2021-03-11T00:37:56.541 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-03-11T00:37:56.543 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:56.580 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:56 +0000 (0:00:00.136) 0:01:33.191 ******** 2021-03-11T00:37:56.623 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:56 +0000 (0:00:00.049) 0:01:33.241 ******** 2021-03-11T00:37:56.674 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2021-03-11T00:37:56.676 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:56.706 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:56 +0000 (0:00:00.083) 0:01:33.324 ******** 2021-03-11T00:37:56.799 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2021-03-11T00:37:56.800 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-03-11T00:37:56.826 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:56 +0000 (0:00:00.119) 0:01:33.444 ******** 2021-03-11T00:37:56.850 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:56.890 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:56 +0000 (0:00:00.064) 0:01:33.508 ******** 2021-03-11T00:37:56.941 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:56.973 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:56 +0000 (0:00:00.082) 0:01:33.591 ******** 2021-03-11T00:37:56.991 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:57.018 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:57 +0000 (0:00:00.045) 0:01:33.636 ******** 2021-03-11T00:37:57.122 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2021-03-11T00:37:57.123 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:57.149 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:57 +0000 (0:00:00.128) 0:01:33.765 ******** 2021-03-11T00:37:57.154 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-11T00:37:57.156 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:57.190 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:57 +0000 (0:00:00.042) 0:01:33.808 ******** 2021-03-11T00:37:57.306 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:57.331 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:57 +0000 (0:00:00.141) 0:01:33.949 ******** 2021-03-11T00:37:57.339 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2021-03-11T00:37:57.341 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:57.407 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:57 +0000 (0:00:00.076) 0:01:34.025 ******** 2021-03-11T00:37:57.524 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2021-03-11T00:37:57.526 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:57.577 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:57 +0000 (0:00:00.169) 0:01:34.195 ******** 2021-03-11T00:37:57.624 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2021-03-11T00:37:57.626 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:57.662 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:57 +0000 (0:00:00.084) 0:01:34.280 ******** 2021-03-11T00:37:57.700 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-03-11T00:37:57.702 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:57.725 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:57 +0000 (0:00:00.063) 0:01:34.343 ******** 2021-03-11T00:37:57.771 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2021-03-11T00:37:57.773 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:57.805 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:57 +0000 (0:00:00.080) 0:01:34.424 ******** 2021-03-11T00:37:57.859 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-03-11T00:37:57.861 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:57.885 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:57 +0000 (0:00:00.079) 0:01:34.503 ******** 2021-03-11T00:37:57.977 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2021-03-11T00:37:57.978 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:58.004 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:58 +0000 (0:00:00.118) 0:01:34.622 ******** 2021-03-11T00:37:58.033 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:58.037 INFO:teuthology.task.ansible.out:TASK [testnode : Install packages] ********************************************* 2021-03-11T00:37:58.039 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:58.065 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:58 +0000 (0:00:00.061) 0:01:34.683 ******** 2021-03-11T00:37:58.104 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:58.105 INFO:teuthology.task.ansible.out:TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-03-11T00:37:58.105 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:58.146 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:58 +0000 (0:00:00.080) 0:01:34.764 ******** 2021-03-11T00:37:58.176 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2021-03-11T00:37:58.181 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:58.210 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:58 +0000 (0:00:00.064) 0:01:34.828 ******** 2021-03-11T00:37:58.232 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-03-11T00:37:58.241 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:58.266 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:58 +0000 (0:00:00.056) 0:01:34.885 ******** 2021-03-11T00:37:58.308 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:58.310 INFO:teuthology.task.ansible.out:TASK [testnode : Remove packages] ********************************************** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:58.338 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:58 +0000 (0:00:00.071) 0:01:34.956 ******** 2021-03-11T00:37:58.377 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:58.415 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:58 +0000 (0:00:00.076) 0:01:35.033 ******** 2021-03-11T00:37:58.441 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-03-11T00:37:58.444 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:58.472 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:58 +0000 (0:00:00.057) 0:01:35.090 ******** 2021-03-11T00:37:58.529 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2021-03-11T00:37:58.531 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:58.558 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:58 +0000 (0:00:00.086) 0:01:35.177 ******** 2021-03-11T00:37:58.604 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2021-03-11T00:37:58.606 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:58.632 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:58 +0000 (0:00:00.073) 0:01:35.251 ******** 2021-03-11T00:37:58.690 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-03-11T00:37:58.692 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:58.722 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:58 +0000 (0:00:00.089) 0:01:35.340 ******** 2021-03-11T00:37:58.753 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2021-03-11T00:37:58.755 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:58.789 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:58 +0000 (0:00:00.067) 0:01:35.407 ******** 2021-03-11T00:37:58.839 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-03-11T00:37:58.841 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:58.869 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:58 +0000 (0:00:00.080) 0:01:35.487 ******** 2021-03-11T00:37:58.879 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2021-03-11T00:37:58.881 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:58.911 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:58 +0000 (0:00:00.041) 0:01:35.529 ******** 2021-03-11T00:37:58.992 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2021-03-11T00:37:58.994 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:59.021 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:59 +0000 (0:00:00.110) 0:01:35.640 ******** 2021-03-11T00:37:59.052 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:59.081 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:59 +0000 (0:00:00.059) 0:01:35.699 ******** 2021-03-11T00:37:59.146 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2021-03-11T00:37:59.148 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:59.185 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:59 +0000 (0:00:00.104) 0:01:35.803 ******** 2021-03-11T00:37:59.188 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:59.207 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2021-03-11T00:37:59.209 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:59.242 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:59 +0000 (0:00:00.056) 0:01:35.859 ******** 2021-03-11T00:37:59.285 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:59.314 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:59 +0000 (0:00:00.072) 0:01:35.932 ******** 2021-03-11T00:37:59.338 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:59.341 INFO:teuthology.task.ansible.out:TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2021-03-11T00:37:59.341 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:59.372 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:59 +0000 (0:00:00.058) 0:01:35.990 ******** 2021-03-11T00:37:59.403 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:59.405 INFO:teuthology.task.ansible.out:TASK [testnode : Install abrt] ************************************************* skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:59.440 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:59 +0000 (0:00:00.067) 0:01:36.058 ******** 2021-03-11T00:37:59.481 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2021-03-11T00:37:59.482 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:59.506 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:59 +0000 (0:00:00.066) 0:01:36.124 ******** 2021-03-11T00:37:59.536 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:59.538 INFO:teuthology.task.ansible.out:TASK [testnode : Enable abrt-auto-reporting] *********************************** skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:59.570 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:59 +0000 (0:00:00.064) 0:01:36.188 ******** 2021-03-11T00:37:59.572 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:59.643 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:59.645 INFO:teuthology.task.ansible.out:TASK [testnode : Restart abrtd] ************************************************ skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:59.669 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:59 +0000 (0:00:00.098) 0:01:36.287 ******** 2021-03-11T00:37:59.677 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2021-03-11T00:37:59.680 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:59.709 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:59 +0000 (0:00:00.039) 0:01:36.327 ******** 2021-03-11T00:37:59.711 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:59.814 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2021-03-11T00:37:59.816 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:59.839 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:59 +0000 (0:00:00.130) 0:01:36.457 ******** 2021-03-11T00:37:59.841 INFO:teuthology.task.ansible.out: 2021-03-11T00:37:59.897 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.005132", "end": "2021-03-11 00:37:59.768241", "rc": 0, "start": "2021-03-11 00:37:59.763109", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-11T00:37:59.921 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:59 +0000 (0:00:00.081) 0:01:36.539 ******** 2021-03-11T00:37:59.929 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2021-03-11T00:37:59.930 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:37:59.966 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:37:59 +0000 (0:00:00.045) 0:01:36.585 ******** 2021-03-11T00:37:59.969 INFO:teuthology.task.ansible.out: 2021-03-11T00:38:00.064 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2021-03-11T00:38:00.090 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:00 +0000 (0:00:00.123) 0:01:36.708 ******** 2021-03-11T00:38:00.207 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2021-03-11T00:38:00.209 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.005068", "end": "2021-03-11 00:38:00.445945", "rc": 0, "start": "2021-03-11 00:38:00.440877", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-11T00:38:00.238 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:00 +0000 (0:00:00.148) 0:01:36.856 ******** 2021-03-11T00:38:00.332 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2021-03-11T00:38:00.357 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:00 +0000 (0:00:00.119) 0:01:36.976 ******** 2021-03-11T00:38:00.530 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2021-03-11T00:38:00.532 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2021-03-11T00:38:00.556 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:00 +0000 (0:00:00.198) 0:01:37.174 ******** 2021-03-11T00:38:00.648 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2021-03-11T00:38:00.650 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:00.672 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:00 +0000 (0:00:00.116) 0:01:37.290 ******** 2021-03-11T00:38:00.789 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2021-03-11T00:38:00.791 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2021-03-11T00:38:00.818 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:00 +0000 (0:00:00.145) 0:01:37.436 ******** 2021-03-11T00:38:00.899 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2021-03-11T00:38:00.901 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:00.928 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:00 +0000 (0:00:00.110) 0:01:37.547 ******** 2021-03-11T00:38:02.090 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-03-11T00:38:02.092 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2021-03-11T00:38:02.123 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2021-03-11T00:38:03.019 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2021-03-11T00:38:03.049 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:03 +0000 (0:00:02.120) 0:01:39.667 ******** 2021-03-11T00:38:03.070 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2021-03-11T00:38:03.099 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:03 +0000 (0:00:00.049) 0:01:39.717 ******** 2021-03-11T00:38:04.036 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2021-03-11T00:38:04.038 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"cache_update_time": 1615422969, "cache_updated": false, "changed": false} 2021-03-11T00:38:04.064 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:04 +0000 (0:00:00.961) 0:01:40.679 ******** 2021-03-11T00:38:04.132 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"cache_update_time": 1615422974, "cache_updated": false, "changed": false} 2021-03-11T00:38:04.157 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:04 +0000 (0:00:00.096) 0:01:40.775 ******** 2021-03-11T00:38:04.164 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2021-03-11T00:38:04.196 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:04 +0000 (0:00:00.038) 0:01:40.814 ******** 2021-03-11T00:38:04.269 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:04 +0000 (0:00:00.072) 0:01:40.887 ******** 2021-03-11T00:38:04.448 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2021-03-11T00:38:04.449 INFO:teuthology.task.ansible.out:[WARNING]: Consider using 'become', 'become_method', and 'become_user' rather than running sudo changed: [smithi119.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.023654", "end": "2021-03-11 00:38:04.322638", "rc": 0, "start": "2021-03-11 00:38:04.298984", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-11T00:38:04.475 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:04 +0000 (0:00:00.206) 0:01:41.093 ******** 2021-03-11T00:38:04.572 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.035048", "end": "2021-03-11 00:38:04.792503", "rc": 0, "start": "2021-03-11 00:38:04.757455", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-11T00:38:04.600 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:04 +0000 (0:00:00.124) 0:01:41.218 ******** 2021-03-11T00:38:07.322 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2021-03-11T00:38:07.324 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"cache_update_time": 1615422969, "cache_updated": false, "changed": false} 2021-03-11T00:38:07.354 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:07 +0000 (0:00:02.752) 0:01:43.970 ******** 2021-03-11T00:38:07.520 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"cache_update_time": 1615422974, "cache_updated": false, "changed": false} 2021-03-11T00:38:07.548 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:07 +0000 (0:00:00.195) 0:01:44.166 ******** 2021-03-11T00:38:17.431 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2021-03-11T00:38:17.440 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 17 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 102064 files and directorie 2021-03-11T00:38:17.442 INFO:teuthology.task.ansible.out:s currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.4) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 17 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Read 2021-03-11T00:38:17.443 INFO:teuthology.task.ansible.out:ing database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 102064 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.4) ..."]} 2021-03-11T00:38:17.499 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:17 +0000 (0:00:09.950) 0:01:54.117 ******** 2021-03-11T00:38:17.509 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 17 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 102064 files and directorie 2021-03-11T00:38:17.511 INFO:teuthology.task.ansible.out:s currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.4) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 17 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Read 2021-03-11T00:38:17.513 INFO:teuthology.task.ansible.out:ing database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 102064 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.4) ..."]} 2021-03-11T00:38:17.556 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:17 +0000 (0:00:00.057) 0:01:54.174 ******** 2021-03-11T00:38:18.527 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* ok: [smithi119.front.sepia.ceph.com] => {"cache_update_time": 1615422969, "cache_updated": false, "changed": false} 2021-03-11T00:38:18.552 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:18 +0000 (0:00:00.995) 0:01:55.170 ******** 2021-03-11T00:38:18.608 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"cache_update_time": 1615422974, "cache_updated": false, "changed": false} 2021-03-11T00:38:18.610 INFO:teuthology.task.ansible.out: 2021-03-11T00:38:18.651 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:18 +0000 (0:00:00.099) 0:01:55.269 ******** 2021-03-11T00:38:18.769 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:18.816 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:18 +0000 (0:00:00.164) 0:01:55.434 ******** 2021-03-11T00:38:18.839 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:18.930 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:18 +0000 (0:00:00.114) 0:01:55.549 ******** 2021-03-11T00:38:29.297 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-03-11T00:38:29.299 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => {"cache_update_time": 1615422969, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk\n gitweb git-cvs git-mediawiki git-svn samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\nThe following packages will be upgraded:\n git\n1 upgraded, 6 newly installed, 0 to remove and 16 not upgraded.\nNeed to get 14.3 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.36 [40.5 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.36 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 git amd64 1:2.17.1-1ubuntu0.8 [3916 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.36 [5176 kB]\nGet:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.36 [870 kB]\nFetched 14.3 MB in 1s (12.1 MB/s)\nSelecting previously unselected package librados2.\r\n(Reading database .. 2021-03-11T00:38:29.301 INFO:teuthology.task.ansible.out:. \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 101922 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...\r\nPreparing to unpack .../4-git_1%3a2.17.1-1ubuntu0.8_amd64.deb ...\r\nUnpacking git (1:2.17.1-1ubuntu0.8) over (1:2.17.1-1ubuntu0.7) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../5-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../6-qemu-utils_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up git (1:2.17.1-1ubuntu0.8) ...\r\nSetti 2021-03-11T00:38:29.303 INFO:teuthology.task.ansible.out:ng up librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.4) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk", " gitweb git-cvs git-mediawiki git-svn samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "The following packages will be upgraded:", " git", "1 upgraded, 6 newly installed, 0 to remove and 16 not upgraded.", "Need to get 14.3 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.36 [40.5 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.36 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 git amd64 1:2.17.1-1ubuntu0.8 [3916 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.36 [5176 kB]", "Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.1 2021-03-11T00:38:29.304 INFO:teuthology.task.ansible.out:1+dfsg-1ubuntu7.36 [870 kB]", "Fetched 14.3 MB in 1s (12.1 MB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 101922 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...", "Preparing to unpack .../4-git_1%3a2.17.1-1ubuntu0.8_amd64.deb ...", "Unpacking git (1:2.17.1-1ubuntu0.8) over (1:2.17.1-1ubuntu0.7) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../5-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../6-qemu-utils_1%3a2.11+dfsg-1ub 2021-03-11T00:38:29.306 INFO:teuthology.task.ansible.out:untu7.36_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up git (1:2.17.1-1ubuntu0.8) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.4) ..."]} 2021-03-11T00:38:29.348 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:29 +0000 (0:00:10.417) 0:02:05.967 ******** 2021-03-11T00:38:30.563 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2021-03-11T00:38:30.565 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"cache_update_time": 1615422969, "cache_updated": false, "changed": false} 2021-03-11T00:38:30.620 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:30 +0000 (0:00:01.271) 0:02:07.238 ******** 2021-03-11T00:38:31.697 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages with --no-install-recommends] **************** 2021-03-11T00:38:31.699 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"cache_update_time": 1615422969, "cache_updated": false, "changed": false} 2021-03-11T00:38:31.727 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:31 +0000 (0:00:01.106) 0:02:08.345 ******** 2021-03-11T00:38:32.098 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2021-03-11T00:38:32.100 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2021-03-11T00:38:32.137 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:32 +0000 (0:00:00.410) 0:02:08.755 ******** 2021-03-11T00:38:32.345 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:32.374 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:32 +0000 (0:00:00.236) 0:02:08.992 ******** 2021-03-11T00:38:32.555 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-03-11T00:38:32.561 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => {"cache_update_time": 1615422974, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk\n gitweb git-cvs git-mediawiki git-svn samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\nThe following packages will be upgraded:\n git\n1 upgraded, 6 newly installed, 0 to remove and 16 not upgraded.\nNeed to get 14.3 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.36 [40.5 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.36 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 git amd64 1:2.17.1-1ubuntu0.8 [3916 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.36 [5176 kB]\nGet:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.36 [870 kB]\nFetched 14.3 MB in 4s (3421 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database .. 2021-03-11T00:38:32.562 INFO:teuthology.task.ansible.out:. \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 101922 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...\r\nPreparing to unpack .../4-git_1%3a2.17.1-1ubuntu0.8_amd64.deb ...\r\nUnpacking git (1:2.17.1-1ubuntu0.8) over (1:2.17.1-1ubuntu0.7) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../5-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../6-qemu-utils_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up git (1:2.17.1-1ubuntu0.8) ...\r\nSetti 2021-03-11T00:38:32.564 INFO:teuthology.task.ansible.out:ng up librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.4) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk", " gitweb git-cvs git-mediawiki git-svn samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "The following packages will be upgraded:", " git", "1 upgraded, 6 newly installed, 0 to remove and 16 not upgraded.", "Need to get 14.3 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.36 [40.5 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.36 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 git amd64 1:2.17.1-1ubuntu0.8 [3916 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.36 [5176 kB]", "Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.1 2021-03-11T00:38:32.566 INFO:teuthology.task.ansible.out:1+dfsg-1ubuntu7.36 [870 kB]", "Fetched 14.3 MB in 4s (3421 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 101922 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...", "Preparing to unpack .../4-git_1%3a2.17.1-1ubuntu0.8_amd64.deb ...", "Unpacking git (1:2.17.1-1ubuntu0.8) over (1:2.17.1-1ubuntu0.7) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../5-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../6-qemu-utils_1%3a2.11+dfsg-1ub 2021-03-11T00:38:32.567 INFO:teuthology.task.ansible.out:untu7.36_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up git (1:2.17.1-1ubuntu0.8) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.4) ..."]} 2021-03-11T00:38:32.606 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:32 +0000 (0:00:00.232) 0:02:09.225 ******** 2021-03-11T00:38:32.807 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2021-03-11T00:38:32.809 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2021-03-11T00:38:32.884 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:32 +0000 (0:00:00.263) 0:02:09.502 ******** 2021-03-11T00:38:33.643 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2021-03-11T00:38:33.645 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2021-03-11T00:38:33.672 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:33 +0000 (0:00:00.788) 0:02:10.290 ******** 2021-03-11T00:38:33.847 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2021-03-11T00:38:33.849 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"cache_update_time": 1615422974, "cache_updated": false, "changed": false} 2021-03-11T00:38:33.883 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:33 +0000 (0:00:00.210) 0:02:10.501 ******** 2021-03-11T00:38:33.997 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2021-03-11T00:38:33.999 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1238, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-03-11T00:38:34.053 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:34 +0000 (0:00:00.170) 0:02:10.671 ******** 2021-03-11T00:38:34.351 INFO:teuthology.task.ansible.out: 2021-03-11T00:38:34.352 INFO:teuthology.task.ansible.out:TASK [testnode : Set up static IP in /etc/hosts.] ****************************** ok: [smithi119.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-03-11T00:38:34.413 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:34 +0000 (0:00:00.360) 0:02:11.031 ******** 2021-03-11T00:38:34.902 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages with --no-install-recommends] **************** 2021-03-11T00:38:34.904 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"cache_update_time": 1615422974, "cache_updated": false, "changed": false} 2021-03-11T00:38:34.936 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:34 +0000 (0:00:00.522) 0:02:11.554 ******** 2021-03-11T00:38:34.966 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-03-11T00:38:34.970 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Thu 2021-03-11 00:35:20 UTC", "ActiveEnterTimestampMonotonic": "17554572", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-journald.socket nss-lookup.target sysinit.target network.target remote-fs.target basic.target system.slice systemd-tmpfiles-setup.service -.mount", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Thu 2021-03-11 00:35:19 UTC", "AssertTimestampMonotonic": "17166556", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Thu 2021-03-11 00:35:19 UTC", "ConditionTimestampMonotonic": "17166556", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2021-03-11T00:38:34.975 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1289", "ExecMainStartTimestamp": "Thu 2021-03-11 00:35:20 UTC", "ExecMainStartTimestampMonotonic": "17554561", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Thu 2021-03-11 00:35:19 UTC", "InactiveExitTimestampMonotonic": "17167462", "InvocationID": "d59ed46da03b4e25adb508aa25e32d09", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2021-03-11T00:38:34.977 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1289", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2021-03-11T00:38:34.978 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Thu 2021-03-11 00:35:20 UTC", "StateChangeTimestampMonotonic": "17554572", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Thu 2021-03-11 00:35:20 UTC", "WatchdogTimestampMonotonic": "17554570", "WatchdogUSec": "0"}} 2021-03-11T00:38:35.062 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:35 +0000 (0:00:00.126) 0:02:11.680 ******** 2021-03-11T00:38:35.244 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2021-03-11T00:38:35.430 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:35 +0000 (0:00:00.193) 0:02:11.874 ******** TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Thursday 11 March 2021 00:38:35 +0000 (0:00:00.087) 0:02:11.962 ******** TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Thursday 11 March 2021 00:38:35 +0000 (0:00:00.082) 0:02:12.044 ******** 2021-03-11T00:38:35.487 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:35.517 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:35 +0000 (0:00:00.086) 0:02:12.131 ******** 2021-03-11T00:38:35.735 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:35.758 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:35 +0000 (0:00:00.245) 0:02:12.377 ******** 2021-03-11T00:38:35.825 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2021-03-11T00:38:35.862 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:35 +0000 (0:00:00.102) 0:02:12.479 ******** 2021-03-11T00:38:35.977 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:36.027 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:36 +0000 (0:00:00.165) 0:02:12.645 ******** 2021-03-11T00:38:36.230 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:36.273 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:36 +0000 (0:00:00.246) 0:02:12.891 ******** 2021-03-11T00:38:36.486 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:36.515 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:36 +0000 (0:00:00.241) 0:02:13.133 ******** 2021-03-11T00:38:36.660 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-03-11T00:38:36.662 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:36.686 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:36 +0000 (0:00:00.170) 0:02:13.303 ******** 2021-03-11T00:38:36.740 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2021-03-11T00:38:36.742 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2021-03-11T00:38:36.791 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:36 +0000 (0:00:00.104) 0:02:13.408 ******** 2021-03-11T00:38:36.863 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-03-11T00:38:36.865 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:36.896 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:36 +0000 (0:00:00.105) 0:02:13.514 ******** 2021-03-11T00:38:36.898 INFO:teuthology.task.ansible.out: 2021-03-11T00:38:37.012 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-11T00:38:37.014 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:37.041 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:37 +0000 (0:00:00.145) 0:02:13.659 ******** 2021-03-11T00:38:37.065 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2021-03-11T00:38:37.067 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1238, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-03-11T00:38:37.100 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:37 +0000 (0:00:00.056) 0:02:13.716 ******** 2021-03-11T00:38:37.148 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-03-11T00:38:37.149 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:37.174 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:37 +0000 (0:00:00.075) 0:02:13.792 ******** 2021-03-11T00:38:37.422 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-03-11T00:38:37.424 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:37.449 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:37 +0000 (0:00:00.275) 0:02:14.067 ******** 2021-03-11T00:38:37.456 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2021-03-11T00:38:37.458 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-03-11T00:38:37.525 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:37 +0000 (0:00:00.076) 0:02:14.144 ******** 2021-03-11T00:38:37.650 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-03-11T00:38:37.652 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:37.681 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:37 +0000 (0:00:00.155) 0:02:14.299 ******** 2021-03-11T00:38:37.878 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-03-11T00:38:37.880 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:37.910 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:37 +0000 (0:00:00.228) 0:02:14.529 ******** 2021-03-11T00:38:38.082 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:38.112 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:38 +0000 (0:00:00.201) 0:02:14.730 ******** 2021-03-11T00:38:38.124 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-03-11T00:38:38.136 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Thu 2021-03-11 00:35:29 UTC", "ActiveEnterTimestampMonotonic": "18031522", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "remote-fs.target sysinit.target systemd-tmpfiles-setup.service system.slice -.mount systemd-journald.socket network.target nss-lookup.target basic.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Thu 2021-03-11 00:35:29 UTC", "AssertTimestampMonotonic": "17706283", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Thu 2021-03-11 00:35:29 UTC", "ConditionTimestampMonotonic": "17706283", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2021-03-11T00:38:38.138 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1291", "ExecMainStartTimestamp": "Thu 2021-03-11 00:35:29 UTC", "ExecMainStartTimestampMonotonic": "18031510", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Thu 2021-03-11 00:35:29 UTC", "InactiveExitTimestampMonotonic": "17706884", "InvocationID": "fa686bdb59c74246a8dd637d4c3581c5", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2021-03-11T00:38:38.140 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128028", "LimitNPROCSoft": "128028", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128028", "LimitSIGPENDINGSoft": "128028", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1291", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2021-03-11T00:38:38.141 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Thu 2021-03-11 00:35:29 UTC", "StateChangeTimestampMonotonic": "18031522", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Thu 2021-03-11 00:35:29 UTC", "WatchdogTimestampMonotonic": "18031521", "WatchdogUSec": "0"}} 2021-03-11T00:38:38.175 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:38 +0000 (0:00:00.063) 0:02:14.793 ******** 2021-03-11T00:38:38.365 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:38.395 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:38 +0000 (0:00:00.219) 0:02:15.013 ******** 2021-03-11T00:38:38.404 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-03-11T00:38:38.406 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:38.454 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:38 +0000 (0:00:00.058) 0:02:15.072 ******** 2021-03-11T00:38:38.617 INFO:teuthology.task.ansible.out: 2021-03-11T00:38:38.621 INFO:teuthology.task.ansible.out:TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-03-11T00:38:38.628 INFO:teuthology.task.ansible.out: skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:38.632 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:38.656 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:38 +0000 (0:00:00.202) 0:02:15.274 ******** 2021-03-11T00:38:38.693 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:38 +0000 (0:00:00.036) 0:02:15.311 ******** 2021-03-11T00:38:38.775 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-03-11T00:38:38.779 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:38.801 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:38 +0000 (0:00:00.107) 0:02:15.419 ******** 2021-03-11T00:38:38.810 INFO:teuthology.task.ansible.out: 2021-03-11T00:38:38.817 INFO:teuthology.task.ansible.out:TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-03-11T00:38:38.819 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:38.860 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:38 +0000 (0:00:00.059) 0:02:15.479 ******** 2021-03-11T00:38:38.910 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-03-11T00:38:38.912 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:38.990 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:38 +0000 (0:00:00.129) 0:02:15.609 ******** 2021-03-11T00:38:39.017 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-03-11T00:38:39.019 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:39.055 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:39 +0000 (0:00:00.064) 0:02:15.673 ******** 2021-03-11T00:38:39.095 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-03-11T00:38:39.097 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:39.169 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:39 +0000 (0:00:00.113) 0:02:15.787 ******** 2021-03-11T00:38:39.229 INFO:teuthology.task.ansible.out: 2021-03-11T00:38:39.230 INFO:teuthology.task.ansible.out:TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:39.259 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:39 +0000 (0:00:00.090) 0:02:15.877 ******** 2021-03-11T00:38:39.397 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-03-11T00:38:39.413 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:39.436 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:39 +0000 (0:00:00.176) 0:02:16.054 ******** 2021-03-11T00:38:39.497 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-03-11T00:38:39.499 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:39.523 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:39 +0000 (0:00:00.087) 0:02:16.142 ******** 2021-03-11T00:38:39.642 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-03-11T00:38:39.647 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:39.678 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:39 +0000 (0:00:00.154) 0:02:16.297 ******** 2021-03-11T00:38:39.750 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-03-11T00:38:39.758 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:39.780 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:39 +0000 (0:00:00.100) 0:02:16.398 ******** 2021-03-11T00:38:39.889 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-03-11T00:38:39.891 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:39.913 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:39 +0000 (0:00:00.133) 0:02:16.531 ******** 2021-03-11T00:38:39.917 INFO:teuthology.task.ansible.out: 2021-03-11T00:38:39.925 INFO:teuthology.task.ansible.out: 2021-03-11T00:38:39.938 INFO:teuthology.task.ansible.out:TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-03-11T00:38:39.940 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:39.987 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:39 +0000 (0:00:00.073) 0:02:16.605 ******** 2021-03-11T00:38:40.089 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-11T00:38:40.091 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:40.116 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:40 +0000 (0:00:00.129) 0:02:16.734 ******** 2021-03-11T00:38:40.155 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-03-11T00:38:40.157 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:40.194 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:40 +0000 (0:00:00.077) 0:02:16.812 ******** 2021-03-11T00:38:40.307 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-03-11T00:38:40.309 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:40.352 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:40 +0000 (0:00:00.157) 0:02:16.970 ******** 2021-03-11T00:38:40.369 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-03-11T00:38:40.370 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:40.410 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:40 +0000 (0:00:00.057) 0:02:17.028 ******** 2021-03-11T00:38:40.524 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-03-11T00:38:40.529 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:40.565 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:40 +0000 (0:00:00.155) 0:02:17.184 ******** 2021-03-11T00:38:40.582 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:40.614 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:40 +0000 (0:00:00.048) 0:02:17.232 ******** 2021-03-11T00:38:40.628 INFO:teuthology.task.ansible.out: 2021-03-11T00:38:40.821 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-03-11T00:38:40.824 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:40.851 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:40 +0000 (0:00:00.236) 0:02:17.469 ******** 2021-03-11T00:38:40.902 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2021-03-11T00:38:40.904 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:40.958 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:40 +0000 (0:00:00.107) 0:02:17.576 ******** 2021-03-11T00:38:40.990 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-03-11T00:38:40.993 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:41.060 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:41 +0000 (0:00:00.101) 0:02:17.678 ******** 2021-03-11T00:38:41.170 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-03-11T00:38:41.172 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:41.203 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:41 +0000 (0:00:00.142) 0:02:17.821 ******** 2021-03-11T00:38:41.301 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2021-03-11T00:38:41.303 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2021-03-11T00:38:41.328 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:41 +0000 (0:00:00.125) 0:02:17.946 ******** 2021-03-11T00:38:41.434 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-03-11T00:38:41.436 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:41.483 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:41 +0000 (0:00:00.155) 0:02:18.102 ******** 2021-03-11T00:38:41.772 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-03-11T00:38:41.775 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:41.803 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:41 +0000 (0:00:00.319) 0:02:18.421 ******** 2021-03-11T00:38:41.807 INFO:teuthology.task.ansible.out: 2021-03-11T00:38:41.997 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:42.045 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:42 +0000 (0:00:00.241) 0:02:18.663 ******** 2021-03-11T00:38:42.052 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-03-11T00:38:42.054 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2021-03-11T00:38:42.058 INFO:teuthology.task.ansible.out: 2021-03-11T00:38:42.118 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:42 +0000 (0:00:00.072) 0:02:18.736 ******** 2021-03-11T00:38:42.203 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-03-11T00:38:42.205 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:42.241 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:42 +0000 (0:00:00.123) 0:02:18.859 ******** 2021-03-11T00:38:42.333 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-03-11T00:38:42.335 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:42.376 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:42 +0000 (0:00:00.135) 0:02:18.995 ******** 2021-03-11T00:38:42.444 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:42.475 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:42 +0000 (0:00:00.098) 0:02:19.093 ******** 2021-03-11T00:38:42.494 INFO:teuthology.task.ansible.out: 2021-03-11T00:38:42.621 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-03-11T00:38:42.622 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:42.645 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:42 +0000 (0:00:00.169) 0:02:19.263 ******** 2021-03-11T00:38:42.835 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-03-11T00:38:42.840 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:42.860 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:42 +0000 (0:00:00.215) 0:02:19.478 ******** 2021-03-11T00:38:42.979 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:43.018 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:43 +0000 (0:00:00.158) 0:02:19.636 ******** 2021-03-11T00:38:43.126 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2021-03-11T00:38:43.134 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2021-03-11T00:38:43.183 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:43 +0000 (0:00:00.164) 0:02:19.801 ******** 2021-03-11T00:38:43.223 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-03-11T00:38:43.225 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:43.260 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:43 +0000 (0:00:00.077) 0:02:19.878 ******** 2021-03-11T00:38:43.466 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:43.488 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:43 +0000 (0:00:00.227) 0:02:20.106 ******** 2021-03-11T00:38:43.610 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-03-11T00:38:43.612 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:43.636 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:43 +0000 (0:00:00.146) 0:02:20.253 ******** 2021-03-11T00:38:43.778 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2021-03-11T00:38:43.780 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} TASK [testnode : Set grub config.] ********************************************* skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:43.840 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:43 +0000 (0:00:00.205) 0:02:20.458 ******** 2021-03-11T00:38:43.876 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:43 +0000 (0:00:00.036) 0:02:20.495 ******** 2021-03-11T00:38:44.141 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2021-03-11T00:38:44.143 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2021-03-11T00:38:44.166 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:44 +0000 (0:00:00.289) 0:02:20.784 ******** 2021-03-11T00:38:44.397 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2021-03-11T00:38:44.408 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2021-03-11T00:38:44.422 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:44 +0000 (0:00:00.256) 0:02:21.041 ******** 2021-03-11T00:38:44.701 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2021-03-11T00:38:44.702 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-03-11T00:38:44.728 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:44 +0000 (0:00:00.305) 0:02:21.347 ******** 2021-03-11T00:38:44.754 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-03-11T00:38:44.756 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2021-03-11T00:38:44.780 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:44 +0000 (0:00:00.051) 0:02:21.399 ******** 2021-03-11T00:38:44.829 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2021-03-11T00:38:44.831 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:44.855 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:44 +0000 (0:00:00.074) 0:02:21.473 ******** 2021-03-11T00:38:44.899 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-03-11T00:38:44.901 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:44.928 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:44 +0000 (0:00:00.073) 0:02:21.546 ******** 2021-03-11T00:38:44.993 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:45.020 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:45 +0000 (0:00:00.088) 0:02:21.635 ******** 2021-03-11T00:38:45.132 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2021-03-11T00:38:45.133 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:45.155 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:45 +0000 (0:00:00.138) 0:02:21.773 ******** 2021-03-11T00:38:45.273 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2021-03-11T00:38:45.276 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:45.316 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:45 +0000 (0:00:00.160) 0:02:21.934 ******** 2021-03-11T00:38:45.404 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2021-03-11T00:38:45.408 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:45.429 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:45 +0000 (0:00:00.112) 0:02:22.047 ******** 2021-03-11T00:38:45.576 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-11T00:38:45.578 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-03-11T00:38:45.600 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:45 +0000 (0:00:00.171) 0:02:22.219 ******** 2021-03-11T00:38:45.630 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2021-03-11T00:38:45.631 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2021-03-11T00:38:45.670 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:45 +0000 (0:00:00.069) 0:02:22.288 ******** 2021-03-11T00:38:45.776 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-03-11T00:38:45.778 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2021-03-11T00:38:45.799 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2021-03-11T00:38:45.813 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2021-03-11T00:38:45.814 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2021-03-11T00:38:45.824 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2021-03-11T00:38:45.835 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2021-03-11T00:38:45.837 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2021-03-11T00:38:45.847 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2021-03-11T00:38:45.848 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2021-03-11T00:38:45.858 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2021-03-11T00:38:45.869 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2021-03-11T00:38:45.880 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2021-03-11T00:38:45.902 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2021-03-11T00:38:45.913 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-03-11T00:38:46.005 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:46 +0000 (0:00:00.335) 0:02:22.624 ******** 2021-03-11T00:38:46.289 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2021-03-11T00:38:46.291 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2021-03-11T00:38:46.327 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:46 +0000 (0:00:00.321) 0:02:22.945 ******** 2021-03-11T00:38:46.496 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2021-03-11T00:38:46.497 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.040872", "end": "2021-03-11 00:38:46.437654", "rc": 0, "start": "2021-03-11 00:38:46.396782", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_b6eecfe89a4363750d696160843a096384b357a8/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_b6eecfe89a4363750d696160843a096384b357a8/virtualenv/bin/teuthology-lock"]} 2021-03-11T00:38:46.535 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:46 +0000 (0:00:00.208) 0:02:23.154 ******** 2021-03-11T00:38:46.654 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:46.685 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:46 +0000 (0:00:00.149) 0:02:23.304 ******** 2021-03-11T00:38:46.835 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2021-03-11T00:38:46.837 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:46.867 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:46 +0000 (0:00:00.181) 0:02:23.485 ******** 2021-03-11T00:38:46.909 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2021-03-11T00:38:46.911 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2021-03-11T00:38:46.939 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:46 +0000 (0:00:00.071) 0:02:23.557 ******** 2021-03-11T00:38:47.006 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:47.043 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:47 +0000 (0:00:00.103) 0:02:23.660 ******** 2021-03-11T00:38:47.192 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2021-03-11T00:38:47.194 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-03-11T00:38:47.227 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:47 +0000 (0:00:00.184) 0:02:23.845 ******** 2021-03-11T00:38:47.366 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2021-03-11T00:38:47.367 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:47.397 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:47 +0000 (0:00:00.170) 0:02:24.016 ******** 2021-03-11T00:38:47.621 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2021-03-11T00:38:47.623 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:47.649 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:47 +0000 (0:00:00.250) 0:02:24.266 ******** 2021-03-11T00:38:47.739 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2021-03-11T00:38:47.741 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:47.775 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:47 +0000 (0:00:00.126) 0:02:24.393 ******** 2021-03-11T00:38:47.889 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2021-03-11T00:38:47.890 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:47.923 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:47 +0000 (0:00:00.147) 0:02:24.541 ******** 2021-03-11T00:38:48.022 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2021-03-11T00:38:48.024 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:48.050 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:48 +0000 (0:00:00.127) 0:02:24.668 ******** 2021-03-11T00:38:48.143 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2021-03-11T00:38:48.145 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"cache_update_time": 1615422969, "cache_updated": false, "changed": false} 2021-03-11T00:38:48.174 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:48 +0000 (0:00:00.123) 0:02:24.792 ******** 2021-03-11T00:38:48.182 INFO:teuthology.task.ansible.out: 2021-03-11T00:38:48.186 INFO:teuthology.task.ansible.out:TASK [testnode : set_fact] ***************************************************** 2021-03-11T00:38:48.188 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-03-11T00:38:48.217 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:48 +0000 (0:00:00.042) 0:02:24.835 ******** 2021-03-11T00:38:48.324 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:48.349 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:48 +0000 (0:00:00.131) 0:02:24.967 ******** 2021-03-11T00:38:48.370 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-03-11T00:38:48.372 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2021-03-11T00:38:48.373 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2021-03-11T00:38:48.374 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2021-03-11T00:38:48.375 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2021-03-11T00:38:48.377 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2021-03-11T00:38:48.380 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} ok: [smithi124.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} skipping: [smithi124.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2021-03-11T00:38:48.390 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2021-03-11T00:38:48.392 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2021-03-11T00:38:48.402 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2021-03-11T00:38:48.414 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2021-03-11T00:38:48.425 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2021-03-11T00:38:48.438 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-03-11T00:38:48.473 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:48 +0000 (0:00:00.124) 0:02:25.091 ******** 2021-03-11T00:38:48.520 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-03-11T00:38:48.522 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item={'block_used': 13643756, 'uuid': 'c2492a7c-537b-4f45-893c-50f733187038', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226451352, 'size_available': 927544737792, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 113943, 'block_size': 4096, 'inode_available': 60941033}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226451352, "block_size": 4096, "block_total": 240095108, "block_used": 13643756, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60941033, "inode_total": 61054976, "inode_used": 113943, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927544737792, "size_total": 983429562368, "uuid": "c2492a7c-537b-4f45-893c-50f733187038"}} 2021-03-11T00:38:48.571 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:48 +0000 (0:00:00.097) 0:02:25.189 ******** 2021-03-11T00:38:48.864 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2021-03-11T00:38:48.866 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.011081", "end": "2021-03-11 00:38:48.737824", "rc": 0, "start": "2021-03-11 00:38:48.726743", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-11T00:38:48.894 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:48 +0000 (0:00:00.323) 0:02:25.512 ******** 2021-03-11T00:38:48.949 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2021-03-11T00:38:48.951 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.015911", "end": "2021-03-11 00:38:48.883724", "rc": 0, "start": "2021-03-11 00:38:48.867813", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_b6eecfe89a4363750d696160843a096384b357a8/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_b6eecfe89a4363750d696160843a096384b357a8/virtualenv/bin/teuthology-lock"]} 2021-03-11T00:38:48.991 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:48 +0000 (0:00:00.096) 0:02:25.609 ******** 2021-03-11T00:38:49.017 INFO:teuthology.task.ansible.out: 2021-03-11T00:38:49.024 INFO:teuthology.task.ansible.out:TASK [testnode : Unmount any non-root mountpoints] ***************************** 2021-03-11T00:38:49.026 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'block_used': 13643756, 'uuid': 'c2492a7c-537b-4f45-893c-50f733187038', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226451352, 'size_available': 927544737792, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 113943, 'block_size': 4096, 'inode_available': 60941033}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226451352, "block_size": 4096, "block_total": 240095108, "block_used": 13643756, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60941033, "inode_total": 61054976, "inode_used": 113943, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927544737792, "size_total": 983429562368, "uuid": "c2492a7c-537b-4f45-893c-50f733187038"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:49.078 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:49 +0000 (0:00:00.087) 0:02:25.697 ******** 2021-03-11T00:38:49.123 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2021-03-11T00:38:49.125 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:49.172 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:49 +0000 (0:00:00.093) 0:02:25.790 ******** 2021-03-11T00:38:49.239 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-03-11T00:38:49.272 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:49 +0000 (0:00:00.099) 0:02:25.890 ******** 2021-03-11T00:38:49.328 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2021-03-11T00:38:49.331 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:49.364 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:49 +0000 (0:00:00.091) 0:02:25.982 ******** 2021-03-11T00:38:49.669 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** skipping: [smithi119.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:49.718 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:49.745 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:49 +0000 (0:00:00.381) 0:02:26.364 ******** 2021-03-11T00:38:50.925 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* ok: [smithi124.front.sepia.ceph.com] => {"cache_update_time": 1615422974, "cache_updated": false, "changed": false} 2021-03-11T00:38:50.965 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:50 +0000 (0:00:01.219) 0:02:27.584 ******** 2021-03-11T00:38:51.127 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2021-03-11T00:38:51.129 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:51.166 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:51 +0000 (0:00:00.199) 0:02:27.784 ******** 2021-03-11T00:38:51.367 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-03-11T00:38:51.369 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item={'block_used': 13643756, 'uuid': 'c2492a7c-537b-4f45-893c-50f733187038', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226451352, 'size_available': 927544737792, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 113943, 'block_size': 4096, 'inode_available': 60941033}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226451352, "block_size": 4096, "block_total": 240095108, "block_used": 13643756, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60941033, "inode_total": 61054976, "inode_used": 113943, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927544737792, "size_total": 983429562368, "uuid": "c2492a7c-537b-4f45-893c-50f733187038"}} 2021-03-11T00:38:51.399 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:51 +0000 (0:00:00.232) 0:02:28.016 ******** 2021-03-11T00:38:51.454 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-03-11T00:38:51.455 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-F2SOZH5rekaITKqSXPr7vJ0YKS1xRKc2EPCQbap6zC0aPCs9rBfD1P0ic53KlD0M'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.597201", "end": "2021-03-11 00:38:51.323876", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-F2SOZH5rekaITKqSXPr7vJ0YKS1xRKc2EPCQbap6zC0aPCs9rBfD1P0ic53KlD0M"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-11 00:38:49.726675", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-11T00:38:51.468 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:51.502 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:51.761 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** changed: [smithi124.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.011096", "end": "2021-03-11 00:38:51.996954", "rc": 0, "start": "2021-03-11 00:38:51.985858", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-11T00:38:51.788 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:51 +0000 (0:00:00.389) 0:02:28.405 ******** 2021-03-11T00:38:52.033 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2021-03-11T00:38:52.035 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'block_used': 13643756, 'uuid': 'c2492a7c-537b-4f45-893c-50f733187038', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226451352, 'size_available': 927544737792, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 113943, 'block_size': 4096, 'inode_available': 60941033}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226451352, "block_size": 4096, "block_total": 240095108, "block_used": 13643756, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60941033, "inode_total": 61054976, "inode_used": 113943, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927544737792, "size_total": 983429562368, "uuid": "c2492a7c-537b-4f45-893c-50f733187038"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:52.058 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:52 +0000 (0:00:00.270) 0:02:28.676 ******** 2021-03-11T00:38:52.065 INFO:teuthology.task.ansible.out: 2021-03-11T00:38:52.210 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-03-11T00:38:52.234 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:52 +0000 (0:00:00.176) 0:02:28.853 ******** 2021-03-11T00:38:52.581 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-03-11T00:38:52.583 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:52.802 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-F2SOZH5rekaITKqSXPr7vJ0YKS1xRKc2aOmOI0Yb7WXktcpE7ndUbTA52u5E3XpL'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.124873", "end": "2021-03-11 00:38:52.664452", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-F2SOZH5rekaITKqSXPr7vJ0YKS1xRKc2aOmOI0Yb7WXktcpE7ndUbTA52u5E3XpL"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-11 00:38:51.539579", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-11T00:38:52.825 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5L6V0', 'wwn-0x5000c500918d7b11'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'c2492a7c-537b-4f45-893c-50f733187038', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5L6V0-part1', 'wwn-0x5000c500918d7b11-part1'], 'uuids': ['c2492a7c-537b-4f45-893c-50f733187038']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c500918d7b11', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5L6V0', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5L6V0", "wwn-0x5000c500918d7b11"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5L6V0-part1", "wwn-0x5000c500918d7b11-part1"], "labels": [], "masters": [], "uuids": ["c2492a7c-537b-4f45-893c-50f733187038"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "c2492a7c-537b-4f45-893c-50f733187038"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5L6V0", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500918d7b11"}}, "skip_reason": 2021-03-11T00:38:52.827 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2021-03-11T00:38:54.207 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-WJ2Dwd-T1u7-zl2k-F6KZ-H7sf-WLr5-anuRJn', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400Y1400BGN', 'nvme-nvme.8086-50484654363230343030593134303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.197923", "end": "2021-03-11 00:38:54.063806", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-WJ2Dwd-T1u7-zl2k-F6KZ-H7sf-WLr5-anuRJn", "nvme-INTEL_SSDPEDMD400G4_PHFT620400Y1400BGN", "nvme-nvme.8086-50484654363230343030593134303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-11 00:38:52.865883", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2021-03-11T00:38:54.209 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-11T00:38:54.241 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:54.256 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:54.407 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-zSg6F72wdSdn9BNdXMXW4ddCCB2k2Tra9tsTefA5IYsjTfM1sVbuqxog4KJJ8TZM'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.693332", "end": "2021-03-11 00:38:54.627067", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-zSg6F72wdSdn9BNdXMXW4ddCCB2k2Tra9tsTefA5IYsjTfM1sVbuqxog4KJJ8TZM"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-11 00:38:52.933735", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-11T00:38:54.440 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:54.452 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:55.591 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-F2SOZH5rekaITKqSXPr7vJ0YKS1xRKc2QuR5Ub4IoFk26YuvcLsOxqgm9XNXInyt'], 'uuids': ['1e70cd5b-9d0b-4921-bbc8-205da15f46df']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.154421", "end": "2021-03-11 00:38:55.456935", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-F2SOZH5rekaITKqSXPr7vJ0YKS1xRKc2QuR5Ub4IoFk26YuvcLsOxqgm9XNXInyt"], "labels": [], "masters": [], "uuids": ["1e70cd5b-9d0b-4921-bbc8-205da15f46df"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-11 00:38:54.302514", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-11T00:38:55.625 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:55.647 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:55.797 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-zSg6F72wdSdn9BNdXMXW4ddCCB2k2TraYApo882bdUht5Hm0ANGCU2yXLsweODvH'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.125332", "end": "2021-03-11 00:38:56.013486", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-zSg6F72wdSdn9BNdXMXW4ddCCB2k2TraYApo882bdUht5Hm0ANGCU2yXLsweODvH"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-11 00:38:54.888154", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} skipping: [smithi124.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', ' 2021-03-11T00:38:55.799 INFO:teuthology.task.ansible.out:vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5PE7R', 'wwn-0x5000c50091d1562a'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'c2492a7c-537b-4f45-893c-50f733187038', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5PE7R-part1', 'wwn-0x5000c50091d1562a-part1'], 'uuids': ['c2492a7c-537b-4f45-893c-50f733187038']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50091d1562a', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5PE7R', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5PE7R", "wwn-0x5000c50091d1562a"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5PE7R-part1", "wwn-0x5000c50091d1562a-part1"], "labels": [], "masters": [], "uuids": ["c2492a7c-537b-4f45-893c-50f733187038"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "c2492a7c-537b-4f45-893c-50f733187038"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5PE7R", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091d1562a"}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:56.959 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-F2SOZH5rekaITKqSXPr7vJ0YKS1xRKc2gCbDJ3WbQfYkJdxMlhcv23kmuhIahDHd'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.234237", "end": "2021-03-11 00:38:56.929512", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-F2SOZH5rekaITKqSXPr7vJ0YKS1xRKc2gCbDJ3WbQfYkJdxMlhcv23kmuhIahDHd"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-11 00:38:55.695275", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-11T00:38:56.971 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:57.355 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-JbkgAX-ucLH-W8ah-yUsY-63f4-y7oM-W5rpj3', 'nvme-INTEL_SSDPEDMD400G4_PHFT620401B8400BGN', 'nvme-nvme.8086-50484654363230343031423834303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.377669", "end": "2021-03-11 00:38:57.579692", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-JbkgAX-ucLH-W8ah-yUsY-63f4-y7oM-W5rpj3", "nvme-INTEL_SSDPEDMD400G4_PHFT620401B8400BGN", "nvme-nvme.8086-50484654363230343031423834303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-11 00:38:56.202023", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2021-03-11T00:38:57.357 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-11T00:38:57.387 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:57.409 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:58.279 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-F2SOZH5rekaITKqSXPr7vJ0YKS1xRKc2KLwhpzVViModXROjzplUfZ6PRPW3Aroh'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.118317", "end": "2021-03-11 00:38:58.255038", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-F2SOZH5rekaITKqSXPr7vJ0YKS1xRKc2KLwhpzVViModXROjzplUfZ6PRPW3Aroh"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-11 00:38:57.136721", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-11T00:38:58.330 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:38:58 +0000 (0:00:06.095) 0:02:34.948 ******** 2021-03-11T00:38:58.501 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-03-11T00:38:58.503 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:58.741 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-03-11T00:38:58.742 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-zSg6F72wdSdn9BNdXMXW4ddCCB2k2TraCRc5da8N9h3yeeySr3DvhEv53irAk2p4'], 'uuids': ['d1136fc5-8375-43e6-8621-963889a0fa28']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.142795", "end": "2021-03-11 00:38:58.973421", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-zSg6F72wdSdn9BNdXMXW4ddCCB2k2TraCRc5da8N9h3yeeySr3DvhEv53irAk2p4"], "labels": [], "masters": [], "uuids": ["d1136fc5-8375-43e6-8621-963889a0fa28"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-11 00:38:57.830626", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-11T00:38:58.773 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:58.785 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:58.828 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-03-11T00:38:58.830 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-F2SOZH5rekaITKqSXPr7vJ0YKS1xRKc2EPCQbap6zC0aPCs9rBfD1P0ic53KlD0M'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.157110", "end": "2021-03-11 00:38:58.797446", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-F2SOZH5rekaITKqSXPr7vJ0YKS1xRKc2EPCQbap6zC0aPCs9rBfD1P0ic53KlD0M"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-11 00:38:58.640336", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.152296 s, 757 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.152296 s, 757 MB/s"], "stdout": "", "stdout_lines": []} 2021-03-11T00:38:58.841 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:58.887 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:59.171 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-F2SOZH5rekaITKqSXPr7vJ0YKS1xRKc2aOmOI0Yb7WXktcpE7ndUbTA52u5E3XpL'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.159135", "end": "2021-03-11 00:38:59.144769", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-F2SOZH5rekaITKqSXPr7vJ0YKS1xRKc2aOmOI0Yb7WXktcpE7ndUbTA52u5E3XpL"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-11 00:38:58.985634", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.154407 s, 747 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.154407 s, 747 MB/s"], "stdout": "", "stdout_lines": []} 2021-03-11T00:38:59.205 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5L6V0', 'wwn-0x5000c500918d7b11'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'c2492a7c-537b-4f45-893c-50f733187038', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5L6V0-part1', 'wwn-0x5000c500918d7b11-part1'], 'uuids': ['c2492a7c-537b-4f45-893c-50f733187038']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c500918d7b11', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5L6V0', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5L6V0", "wwn-0x5000c500918d7b11"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5L6V0-part1", "wwn-0x5000c500918d7b11-part1"], "labels": [], "masters": [], "uuids": ["c2492a7c-537b-4f45-893c-50f733187038"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "c2492a7c-537b-4f45-893c-50f733187038"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5L6V0", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500918d7b11"}}, "skip_reason": 2021-03-11T00:38:59.207 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2021-03-11T00:38:59.430 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-WJ2Dwd-T1u7-zl2k-F6KZ-H7sf-WLr5-anuRJn', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400Y1400BGN', 'nvme-nvme.8086-50484654363230343030593134303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.051797", "end": "2021-03-11 00:38:59.393423", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-WJ2Dwd-T1u7-zl2k-F6KZ-H7sf-WLr5-anuRJn", "nvme-INTEL_SSDPEDMD400G4_PHFT620400Y1400BGN", "nvme-nvme.8086-50484654363230343030593134303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-11 00:38:59.341626", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0469694 s, 2.5 GB/s", "std 2021-03-11T00:38:59.432 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0469694 s, 2.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-03-11T00:38:59.442 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:59.465 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:59.761 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-F2SOZH5rekaITKqSXPr7vJ0YKS1xRKc2QuR5Ub4IoFk26YuvcLsOxqgm9XNXInyt'], 'uuids': ['1e70cd5b-9d0b-4921-bbc8-205da15f46df']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.154876", "end": "2021-03-11 00:38:59.730522", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-F2SOZH5rekaITKqSXPr7vJ0YKS1xRKc2QuR5Ub4IoFk26YuvcLsOxqgm9XNXInyt"], "labels": [], "masters": [], "uuids": ["1e70cd5b-9d0b-4921-bbc8-205da15f46df"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-11 00:38:59.575646", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.151184 s, 763 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.151184 s, 763 MB/s"], "stdout": "", "stdout_lines": []} 2021-03-11T00:38:59.773 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:38:59.795 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:00.121 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-F2SOZH5rekaITKqSXPr7vJ0YKS1xRKc2gCbDJ3WbQfYkJdxMlhcv23kmuhIahDHd'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.158862", "end": "2021-03-11 00:39:00.082971", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-F2SOZH5rekaITKqSXPr7vJ0YKS1xRKc2gCbDJ3WbQfYkJdxMlhcv23kmuhIahDHd"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-11 00:38:59.924109", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.153995 s, 749 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.153995 s, 749 MB/s"], "stdout": "", "stdout_lines": []} 2021-03-11T00:39:00.154 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:00.218 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-03-11T00:39:00.220 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-zSg6F72wdSdn9BNdXMXW4ddCCB2k2TraSfvJup2vhT2g6lBvArQF3fT4AxyaZckn'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.212863", "end": "2021-03-11 00:39:00.439229", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-zSg6F72wdSdn9BNdXMXW4ddCCB2k2TraSfvJup2vhT2g6lBvArQF3fT4AxyaZckn"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-11 00:38:59.226366", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-11T00:39:00.244 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:00.513 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-03-11T00:39:00.516 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-F2SOZH5rekaITKqSXPr7vJ0YKS1xRKc2KLwhpzVViModXROjzplUfZ6PRPW3Aroh'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.155937", "end": "2021-03-11 00:39:00.486163", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-F2SOZH5rekaITKqSXPr7vJ0YKS1xRKc2KLwhpzVViModXROjzplUfZ6PRPW3Aroh"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-11 00:39:00.330226", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.151374 s, 762 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.151374 s, 762 MB/s"], "stdout": "", "stdout_lines": []} 2021-03-11T00:39:00.570 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:00 +0000 (0:00:02.239) 0:02:37.188 ******** 2021-03-11T00:39:01.010 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** changed: [smithi119.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.133803", "end": "2021-03-11 00:39:00.897408", "rc": 0, "start": "2021-03-11 00:39:00.763605", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-11T00:39:01.039 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:01 +0000 (0:00:00.468) 0:02:37.656 ******** 2021-03-11T00:39:01.291 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** changed: [smithi119.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.061076", "end": "2021-03-11 00:39:01.271287", "rc": 0, "start": "2021-03-11 00:39:01.210211", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-11T00:39:01.328 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:01 +0000 (0:00:00.289) 0:02:37.946 ******** 2021-03-11T00:39:01.438 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-03-11T00:39:01.537 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:01 +0000 (0:00:00.209) 0:02:38.155 ******** 2021-03-11T00:39:01.580 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-03-11T00:39:01.582 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-zSg6F72wdSdn9BNdXMXW4ddCCB2k2Tra2vrrFYnmxMzkf22OssA0tmq8acDDJJ3W'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.117518", "end": "2021-03-11 00:39:01.798026", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-zSg6F72wdSdn9BNdXMXW4ddCCB2k2Tra2vrrFYnmxMzkf22OssA0tmq8acDDJJ3W"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-11 00:39:00.680508", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-11T00:39:01.656 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:01 +0000 (0:00:00.118) 0:02:38.273 ******** 2021-03-11T00:39:01.664 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2021-03-11T00:39:01.666 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:01.715 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:01 +0000 (0:00:00.056) 0:02:38.330 ******** 2021-03-11T00:39:01.741 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-03-11T00:39:01.743 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:01.764 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:01.788 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:01 +0000 (0:00:00.076) 0:02:38.406 ******** 2021-03-11T00:39:01.847 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-03-11T00:39:01.849 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:01.882 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:01 +0000 (0:00:00.090) 0:02:38.496 ******** 2021-03-11T00:39:01.924 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2021-03-11T00:39:01.925 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:01.957 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:01 +0000 (0:00:00.074) 0:02:38.571 ******** 2021-03-11T00:39:01.964 INFO:teuthology.task.ansible.out: 2021-03-11T00:39:02.109 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-03-11T00:39:02.111 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-zSg6F72wdSdn9BNdXMXW4ddCCB2k2Tra9tsTefA5IYsjTfM1sVbuqxog4KJJ8TZM'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.154982", "end": "2021-03-11 00:39:02.333582", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-zSg6F72wdSdn9BNdXMXW4ddCCB2k2Tra9tsTefA5IYsjTfM1sVbuqxog4KJJ8TZM"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-11 00:39:02.178600", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.150102 s, 768 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.150102 s, 768 MB/s"], "stdout": "", "stdout_lines": []} 2021-03-11T00:39:02.158 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:02.166 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:02.200 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2021-03-11T00:39:02.202 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-03-11T00:39:02.230 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:02 +0000 (0:00:00.277) 0:02:38.849 ******** 2021-03-11T00:39:02.315 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-03-11T00:39:02.317 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'block_used': 13643756, 'uuid': 'c2492a7c-537b-4f45-893c-50f733187038', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226451352, 'size_available': 927544737792, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 113943, 'block_size': 4096, 'inode_available': 60941033}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226451352, "block_size": 4096, "block_total": 240095108, "block_used": 13643756, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60941033, "inode_total": 61054976, "inode_used": 113943, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927544737792, "size_total": 983429562368, "uuid": "c2492a7c-537b-4f45-893c-50f733187038"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:02.340 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:02 +0000 (0:00:00.110) 0:02:38.959 ******** 2021-03-11T00:39:02.411 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-03-11T00:39:02.413 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:02.436 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:02 +0000 (0:00:00.095) 0:02:39.054 ******** 2021-03-11T00:39:02.505 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-03-11T00:39:02.506 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-zSg6F72wdSdn9BNdXMXW4ddCCB2k2TraYApo882bdUht5Hm0ANGCU2yXLsweODvH'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.155727", "end": "2021-03-11 00:39:02.737516", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-zSg6F72wdSdn9BNdXMXW4ddCCB2k2TraYApo882bdUht5Hm0ANGCU2yXLsweODvH"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-11 00:39:02.581789", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.150978 s, 764 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.150978 s, 764 MB/s"], "stdout": "", "stdout_lines": []} TASK [testnode : set_fact] ***************************************************** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:02.507 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-03-11T00:39:02.507 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids' 2021-03-11T00:39:02.509 INFO:teuthology.task.ansible.out:: ['ata-ST1000NM0033-9ZM173_Z1W5PE7R', 'wwn-0x5000c50091d1562a'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'c2492a7c-537b-4f45-893c-50f733187038', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5PE7R-part1', 'wwn-0x5000c50091d1562a-part1'], 'uuids': ['c2492a7c-537b-4f45-893c-50f733187038']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50091d1562a', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5PE7R', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5PE7R", "wwn-0x5000c50091d1562a"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5PE7R-part1", "wwn-0x5000c50091d1562a-part1"], "labels": [], "masters": [], "uuids": ["c2492a7c-537b-4f45-893c-50f733187038"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "c2492a7c-537b-4f45-893c-50f733187038"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5PE7R", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091d1562a"}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:02.539 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:02 +0000 (0:00:00.102) 0:02:39.157 ******** 2021-03-11T00:39:02.603 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-11T00:39:02.604 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:02.629 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:02 +0000 (0:00:00.090) 0:02:39.247 ******** 2021-03-11T00:39:02.685 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:02.723 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:02 +0000 (0:00:00.093) 0:02:39.341 ******** 2021-03-11T00:39:02.739 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-03-11T00:39:02.740 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-JbkgAX-ucLH-W8ah-yUsY-63f4-y7oM-W5rpj3', 'nvme-INTEL_SSDPEDMD400G4_PHFT620401B8400BGN', 'nvme-nvme.8086-50484654363230343031423834303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.053670", "end": "2021-03-11 00:39:02.946466", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-JbkgAX-ucLH-W8ah-yUsY-63f4-y7oM-W5rpj3", "nvme-INTEL_SSDPEDMD400G4_PHFT620401B8400BGN", "nvme-nvme.8086-50484654363230343031423834303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-11 00:39:02.892796", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0488095 s, 2.4 GB/s", "std 2021-03-11T00:39:02.742 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0488095 s, 2.4 GB/s"], "stdout": "", "stdout_lines": []} 2021-03-11T00:39:02.753 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:02.763 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:02.839 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2021-03-11T00:39:02.851 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:02.902 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:02 +0000 (0:00:00.179) 0:02:39.520 ******** 2021-03-11T00:39:02.944 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:02.969 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:02 +0000 (0:00:00.066) 0:02:39.587 ******** 2021-03-11T00:39:03.087 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-03-11T00:39:03.091 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-zSg6F72wdSdn9BNdXMXW4ddCCB2k2TraCRc5da8N9h3yeeySr3DvhEv53irAk2p4'], 'uuids': ['d1136fc5-8375-43e6-8621-963889a0fa28']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.156826", "end": "2021-03-11 00:39:03.315374", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-zSg6F72wdSdn9BNdXMXW4ddCCB2k2TraCRc5da8N9h3yeeySr3DvhEv53irAk2p4"], "labels": [], "masters": [], "uuids": ["d1136fc5-8375-43e6-8621-963889a0fa28"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-11 00:39:03.158548", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.152576 s, 756 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.152576 s, 756 MB/s"], "stdout": "", "stdout_lines": []} 2021-03-11T00:39:03.105 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:03.128 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:03.463 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-zSg6F72wdSdn9BNdXMXW4ddCCB2k2TraSfvJup2vhT2g6lBvArQF3fT4AxyaZckn'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.164930", "end": "2021-03-11 00:39:03.688304", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-zSg6F72wdSdn9BNdXMXW4ddCCB2k2TraSfvJup2vhT2g6lBvArQF3fT4AxyaZckn"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-11 00:39:03.523374", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.16012 s, 720 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.16012 s, 720 MB/s"], "stdout": "", "stdout_lines": []} 2021-03-11T00:39:03.475 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:03.635 INFO:teuthology.task.ansible.out: 2021-03-11T00:39:03.637 INFO:teuthology.task.ansible.out:TASK [testnode : Create volume group(s)] *************************************** changed: [smithi119.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-03-11T00:39:03.664 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:03 +0000 (0:00:00.695) 0:02:40.283 ******** 2021-03-11T00:39:03.783 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-03-11T00:39:03.785 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-zSg6F72wdSdn9BNdXMXW4ddCCB2k2Tra2vrrFYnmxMzkf22OssA0tmq8acDDJJ3W'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.149954", "end": "2021-03-11 00:39:04.012517", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-zSg6F72wdSdn9BNdXMXW4ddCCB2k2Tra2vrrFYnmxMzkf22OssA0tmq8acDDJJ3W"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-11 00:39:03.862563", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.145012 s, 795 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.145012 s, 795 MB/s"], "stdout": "", "stdout_lines": []} 2021-03-11T00:39:03.833 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:03 +0000 (0:00:00.169) 0:02:40.452 ******** 2021-03-11T00:39:04.277 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2021-03-11T00:39:04.278 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.193548", "end": "2021-03-11 00:39:04.495596", "rc": 0, "start": "2021-03-11 00:39:04.302048", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-11T00:39:04.304 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:04 +0000 (0:00:00.470) 0:02:40.922 ******** 2021-03-11T00:39:04.354 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-03-11T00:39:04.355 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-11T00:39:04.613 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2021-03-11T00:39:04.615 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.102305", "end": "2021-03-11 00:39:04.808093", "rc": 0, "start": "2021-03-11 00:39:04.705788", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-11T00:39:04.652 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:04 +0000 (0:00:00.347) 0:02:41.270 ******** 2021-03-11T00:39:04.688 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-03-11T00:39:04.690 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-11T00:39:04.733 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-03-11T00:39:04.760 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:04 +0000 (0:00:00.108) 0:02:41.378 ******** 2021-03-11T00:39:04.858 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2021-03-11T00:39:04.859 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:04.887 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:04 +0000 (0:00:00.127) 0:02:41.505 ******** 2021-03-11T00:39:04.942 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-03-11T00:39:04.944 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-11T00:39:04.980 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2021-03-11T00:39:04.982 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:05.008 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:05 +0000 (0:00:00.120) 0:02:41.626 ******** 2021-03-11T00:39:05.101 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:05.145 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:05 +0000 (0:00:00.122) 0:02:41.748 ******** 2021-03-11T00:39:05.183 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2021-03-11T00:39:05.185 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:05.208 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:05 +0000 (0:00:00.077) 0:02:41.826 ******** 2021-03-11T00:39:05.251 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-03-11T00:39:05.253 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-11T00:39:05.401 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2021-03-11T00:39:05.403 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-03-11T00:39:05.440 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:05 +0000 (0:00:00.232) 0:02:42.059 ******** 2021-03-11T00:39:05.557 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-03-11T00:39:05.559 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-11T00:39:05.584 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:05 +0000 (0:00:00.143) 0:02:42.202 ******** 2021-03-11T00:39:05.628 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ skipping: [smithi124.front.sepia.ceph.com] => (item={'block_used': 13643756, 'uuid': 'c2492a7c-537b-4f45-893c-50f733187038', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226451352, 'size_available': 927544737792, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 113943, 'block_size': 4096, 'inode_available': 60941033}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226451352, "block_size": 4096, "block_total": 240095108, "block_used": 13643756, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60941033, "inode_total": 61054976, "inode_used": 113943, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927544737792, "size_total": 983429562368, "uuid": "c2492a7c-537b-4f45-893c-50f733187038"}, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:05.656 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:05 +0000 (0:00:00.071) 0:02:42.274 ******** 2021-03-11T00:39:05.730 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-03-11T00:39:05.731 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:05.784 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:05 +0000 (0:00:00.127) 0:02:42.402 ******** 2021-03-11T00:39:05.814 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-03-11T00:39:05.816 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-03-11T00:39:05.841 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:05 +0000 (0:00:00.057) 0:02:42.459 ******** 2021-03-11T00:39:05.861 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-11T00:39:05.863 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:05.912 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:05 +0000 (0:00:00.070) 0:02:42.530 ******** 2021-03-11T00:39:06.019 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-11T00:39:06.021 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:06.046 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:06 +0000 (0:00:00.132) 0:02:42.662 ******** 2021-03-11T00:39:06.084 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:06.090 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** changed: [smithi119.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-03-11T00:39:06.126 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:06 +0000 (0:00:00.082) 0:02:42.744 ******** 2021-03-11T00:39:06.220 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-03-11T00:39:06.231 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2021-03-11T00:39:06.234 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:06.269 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:06 +0000 (0:00:00.142) 0:02:42.887 ******** 2021-03-11T00:39:06.361 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:06.384 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:06 +0000 (0:00:00.115) 0:02:43.003 ******** 2021-03-11T00:39:06.409 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-03-11T00:39:06.411 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-03-11T00:39:06.582 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-03-11T00:39:06.625 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:06.656 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:06 +0000 (0:00:00.270) 0:02:43.274 ******** 2021-03-11T00:39:06.743 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:06.784 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:06 +0000 (0:00:00.127) 0:02:43.402 ******** 2021-03-11T00:39:06.845 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** 2021-03-11T00:39:06.847 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:06.869 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:06 +0000 (0:00:00.085) 0:02:43.487 ******** 2021-03-11T00:39:06.911 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2021-03-11T00:39:06.912 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-03-11T00:39:06.944 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:06 +0000 (0:00:00.075) 0:02:43.562 ******** 2021-03-11T00:39:07.096 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2021-03-11T00:39:07.098 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2021-03-11T00:39:07.136 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:07 +0000 (0:00:00.192) 0:02:43.755 ******** 2021-03-11T00:39:07.218 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2021-03-11T00:39:07.220 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-03-11T00:39:07.245 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:07 +0000 (0:00:00.109) 0:02:43.864 ******** 2021-03-11T00:39:07.548 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-03-11T00:39:07.550 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-11T00:39:07.869 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-11T00:39:08.159 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-11T00:39:08.268 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2021-03-11T00:39:08.269 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => {"changed": true} 2021-03-11T00:39:08.321 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:08 +0000 (0:00:01.070) 0:02:44.934 ******** 2021-03-11T00:39:08.492 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-03-11T00:39:08.494 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-11T00:39:08.769 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-11T00:39:08.797 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:08 +0000 (0:00:00.481) 0:02:45.415 ******** 2021-03-11T00:39:09.058 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-03-11T00:39:09.060 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-03-11T00:39:09.083 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:09 +0000 (0:00:00.285) 0:02:45.701 ******** 2021-03-11T00:39:09.113 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2021-03-11T00:39:09.114 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2021-03-11T00:39:09.140 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:09 +0000 (0:00:00.057) 0:02:45.758 ******** 2021-03-11T00:39:09.225 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** 2021-03-11T00:39:09.227 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:09.256 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:09 +0000 (0:00:00.116) 0:02:45.875 ******** 2021-03-11T00:39:09.260 INFO:teuthology.task.ansible.out: 2021-03-11T00:39:09.382 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-03-11T00:39:09.384 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-03-11T00:39:09.394 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:09.430 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:09 +0000 (0:00:00.166) 0:02:46.041 ******** 2021-03-11T00:39:09.472 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-03-11T00:39:09.475 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:09.502 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:09 +0000 (0:00:00.073) 0:02:46.115 ******** 2021-03-11T00:39:09.536 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-03-11T00:39:09.538 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-03-11T00:39:09.548 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-03-11T00:39:09.550 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:09.572 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:09 +0000 (0:00:00.074) 0:02:46.190 ******** 2021-03-11T00:39:09.718 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-03-11T00:39:09.720 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-03-11T00:39:09.779 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2021-03-11T00:39:09.782 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2021-03-11T00:39:09.809 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:09 +0000 (0:00:00.236) 0:02:46.427 ******** 2021-03-11T00:39:09.891 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-03-11T00:39:09.893 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-03-11T00:39:09.903 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:09.932 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:09 +0000 (0:00:00.123) 0:02:46.550 ******** 2021-03-11T00:39:10.028 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* 2021-03-11T00:39:10.032 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:10.056 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:10 +0000 (0:00:00.123) 0:02:46.674 ******** 2021-03-11T00:39:10.069 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2021-03-11T00:39:10.071 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2021-03-11T00:39:10.109 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:10 +0000 (0:00:00.053) 0:02:46.727 ******** 2021-03-11T00:39:10.138 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** 2021-03-11T00:39:10.140 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:10.171 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:10 +0000 (0:00:00.062) 0:02:46.789 ******** 2021-03-11T00:39:10.404 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2021-03-11T00:39:10.406 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2021-03-11T00:39:10.428 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:10 +0000 (0:00:00.257) 0:02:47.047 ******** 2021-03-11T00:39:10.566 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2021-03-11T00:39:10.568 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-03-11T00:39:10.595 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:10 +0000 (0:00:00.166) 0:02:47.213 ******** 2021-03-11T00:39:10.666 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2021-03-11T00:39:10.667 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2021-03-11T00:39:10.702 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:10 +0000 (0:00:00.106) 0:02:47.320 ******** 2021-03-11T00:39:10.996 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* changed: [smithi124.front.sepia.ceph.com] => {"changed": true} 2021-03-11T00:39:11.017 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:11 +0000 (0:00:00.315) 0:02:47.635 ******** 2021-03-11T00:39:11.264 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2021-03-11T00:39:11.297 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:11 +0000 (0:00:00.280) 0:02:47.916 ******** 2021-03-11T00:39:11.309 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* changed: [smithi124.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2021-03-11T00:39:11.356 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:11 +0000 (0:00:00.057) 0:02:47.975 ******** 2021-03-11T00:39:11.458 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:11.527 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:11 +0000 (0:00:00.130) 0:02:48.105 ******** 2021-03-11T00:39:11.588 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** 2021-03-11T00:39:11.590 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:11.620 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:11 +0000 (0:00:00.132) 0:02:48.237 ******** 2021-03-11T00:39:11.694 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:11.722 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:11 +0000 (0:00:00.101) 0:02:48.339 ******** 2021-03-11T00:39:11.799 INFO:teuthology.task.ansible.out: 2021-03-11T00:39:11.802 INFO:teuthology.task.ansible.out:TASK [testnode : Restart cachefilesd] ****************************************** 2021-03-11T00:39:11.808 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:11.885 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:11 +0000 (0:00:00.164) 0:02:48.503 ******** 2021-03-11T00:39:12.135 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2021-03-11T00:39:12.171 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:12 +0000 (0:00:00.286) 0:02:48.790 ******** 2021-03-11T00:39:12.273 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** ok: [smithi119.front.sepia.ceph.com] => {"cache_update_time": 1615422969, "cache_updated": false, "changed": false} 2021-03-11T00:39:12.333 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:12 +0000 (0:00:00.161) 0:02:48.951 ******** 2021-03-11T00:39:12.465 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2021-03-11T00:39:12.467 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2021-03-11T00:39:12.515 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:12 +0000 (0:00:00.181) 0:02:49.133 ******** 2021-03-11T00:39:12.652 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2021-03-11T00:39:12.655 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.044040", "end": "2021-03-11 00:39:12.618031", "rc": 0, "start": "2021-03-11 00:39:12.573991", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2021-03-11T00:39:12.718 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:12 +0000 (0:00:00.183) 0:02:49.337 ******** 2021-03-11T00:39:12.832 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2021-03-11T00:39:12.834 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:12.868 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:12 +0000 (0:00:00.139) 0:02:49.476 ******** 2021-03-11T00:39:12.947 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:13.002 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:12 +0000 (0:00:00.143) 0:02:49.620 ******** 2021-03-11T00:39:13.161 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2021-03-11T00:39:13.200 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:13 +0000 (0:00:00.197) 0:02:49.818 ******** 2021-03-11T00:39:13.773 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2021-03-11T00:39:13.775 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2021-03-11T00:39:13.811 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:13 +0000 (0:00:00.611) 0:02:50.429 ******** 2021-03-11T00:39:13.931 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2021-03-11T00:39:13.932 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"cache_update_time": 1615422969, "cache_updated": false, "changed": false} 2021-03-11T00:39:13.957 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:13 +0000 (0:00:00.145) 0:02:50.575 ******** 2021-03-11T00:39:14.044 INFO:teuthology.task.ansible.out: 2021-03-11T00:39:14.046 INFO:teuthology.task.ansible.out:TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:14.075 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:14 +0000 (0:00:00.117) 0:02:50.693 ******** 2021-03-11T00:39:14.718 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2021-03-11T00:39:14.720 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2021-03-11T00:39:14.758 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:14 +0000 (0:00:00.682) 0:02:51.376 ******** 2021-03-11T00:39:14.822 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** ok: [smithi124.front.sepia.ceph.com] => {"cache_update_time": 1615422974, "cache_updated": false, "changed": false} 2021-03-11T00:39:14.846 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:14 +0000 (0:00:00.089) 0:02:51.465 ******** 2021-03-11T00:39:14.860 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:14.906 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:14 +0000 (0:00:00.059) 0:02:51.525 ******** 2021-03-11T00:39:15.101 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.044822", "end": "2021-03-11 00:39:15.075128", "rc": 0, "start": "2021-03-11 00:39:15.030306", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2021-03-11T00:39:15.133 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:15 +0000 (0:00:00.226) 0:02:51.751 ******** 2021-03-11T00:39:15.206 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2021-03-11T00:39:15.208 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:15.231 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:15 +0000 (0:00:00.097) 0:02:51.849 ******** 2021-03-11T00:39:15.257 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-03-11T00:39:15.260 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Thu 2021-03-11 00:35:19 UTC", "ActiveEnterTimestampMonotonic": "16932428", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "-.mount basic.target network.target systemd-tmpfiles-setup.service systemd-journald.socket system.slice sysinit.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Thu 2021-03-11 00:35:19 UTC", "AssertTimestampMonotonic": "16796069", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Thu 2021-03-11 00:35:19 UTC", "ConditionTimestampMonotonic": "16796069", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service shutdown.target", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2021-03-11T00:39:15.261 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1145", "ExecMainStartTimestamp": "Thu 2021-03-11 00:35:19 UTC", "ExecMainStartTimestampMonotonic": "16932418", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Thu 2021-03-11 00:35:19 UTC", "InactiveExitTimestampMonotonic": "16796755", "InvocationID": "8ee087ebe36444aca180566f1d9cb9d9", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2021-03-11T00:39:15.263 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1145", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount system.slice sysinit.target", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Thu 2021-03-11 00:35:19 UTC", "S 2021-03-11T00:39:15.265 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "16932428", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Thu 2021-03-11 00:35:19 UTC", "WatchdogTimestampMonotonic": "16932427", "WatchdogUSec": "0"}} 2021-03-11T00:39:15.301 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:15 +0000 (0:00:00.070) 0:02:51.919 ******** 2021-03-11T00:39:15.311 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2021-03-11T00:39:15.313 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:15.361 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:15 +0000 (0:00:00.059) 0:02:51.979 ******** 2021-03-11T00:39:15.390 INFO:teuthology.task.ansible.out: 2021-03-11T00:39:15.391 INFO:teuthology.task.ansible.out:TASK [testnode : set_fact] ***************************************************** ok: [smithi119.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-03-11T00:39:15.464 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:15 +0000 (0:00:00.103) 0:02:52.083 ******** 2021-03-11T00:39:15.561 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:15.584 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:15 +0000 (0:00:00.119) 0:02:52.202 ******** 2021-03-11T00:39:15.655 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2021-03-11T00:39:15.656 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:15.681 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:15 +0000 (0:00:00.097) 0:02:52.299 ******** 2021-03-11T00:39:16.354 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2021-03-11T00:39:16.367 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"cache_update_time": 1615422974, "cache_updated": false, "changed": false} 2021-03-11T00:39:16.429 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:16 +0000 (0:00:00.743) 0:02:53.047 ******** 2021-03-11T00:39:16.516 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2021-03-11T00:39:16.518 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:16.557 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:16 +0000 (0:00:00.124) 0:02:53.172 ******** 2021-03-11T00:39:16.667 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2021-03-11T00:39:16.669 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"cache_update_time": 1615422969, "cache_updated": false, "changed": false} 2021-03-11T00:39:16.726 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:16 +0000 (0:00:00.172) 0:02:53.344 ******** 2021-03-11T00:39:16.780 INFO:teuthology.task.ansible.out: 2021-03-11T00:39:16.782 INFO:teuthology.task.ansible.out:TASK [testnode : Install python-pip on zypper based systems.] ****************** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:16.806 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:16 +0000 (0:00:00.080) 0:02:53.424 ******** 2021-03-11T00:39:17.070 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2021-03-11T00:39:17.072 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2021-03-11T00:39:17.099 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:17 +0000 (0:00:00.292) 0:02:53.717 ******** 2021-03-11T00:39:17.468 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2021-03-11T00:39:17.472 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2021-03-11T00:39:17.529 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:17 +0000 (0:00:00.412) 0:02:54.129 ******** 2021-03-11T00:39:17.610 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2021-03-11T00:39:17.612 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:17.634 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:17 +0000 (0:00:00.122) 0:02:54.252 ******** 2021-03-11T00:39:17.912 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2021-03-11T00:39:17.913 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2021-03-11T00:39:17.939 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:17 +0000 (0:00:00.304) 0:02:54.557 ******** 2021-03-11T00:39:17.986 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-03-11T00:39:17.987 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Thu 2021-03-11 00:35:29 UTC", "ActiveEnterTimestampMonotonic": "17332160", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "basic.target -.mount systemd-tmpfiles-setup.service system.slice systemd-journald.socket sysinit.target network.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Thu 2021-03-11 00:35:28 UTC", "AssertTimestampMonotonic": "17204117", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Thu 2021-03-11 00:35:28 UTC", "ConditionTimestampMonotonic": "17204116", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service shutdown.target", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2021-03-11T00:39:17.989 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1149", "ExecMainStartTimestamp": "Thu 2021-03-11 00:35:29 UTC", "ExecMainStartTimestampMonotonic": "17332150", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Thu 2021-03-11 00:35:28 UTC", "InactiveExitTimestampMonotonic": "17205022", "InvocationID": "925f8dc563d6445b8bd02533f1f902e6", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128028", "LimitNPROCSoft": "128028", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2021-03-11T00:39:17.990 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128028", "LimitSIGPENDINGSoft": "128028", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1149", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount system.slice sysinit.target", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Thu 2021-03-11 00:35:29 UTC", "S 2021-03-11T00:39:17.992 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "17332160", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Thu 2021-03-11 00:35:29 UTC", "WatchdogTimestampMonotonic": "17332159", "WatchdogUSec": "0"}} 2021-03-11T00:39:18.081 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:18 +0000 (0:00:00.141) 0:02:54.699 ******** 2021-03-11T00:39:18.196 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-11T00:39:18.198 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-03-11T00:39:18.314 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:18 +0000 (0:00:00.233) 0:02:54.932 ******** 2021-03-11T00:39:18.407 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-11T00:39:18.409 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:18.444 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:18 +0000 (0:00:00.129) 0:02:55.062 ******** 2021-03-11T00:39:18.515 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2021-03-11T00:39:18.517 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:18.568 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:18 +0000 (0:00:00.123) 0:02:55.186 ******** 2021-03-11T00:39:18.576 INFO:teuthology.task.ansible.out: 2021-03-11T00:39:19.604 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2021-03-11T00:39:19.606 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"cache_update_time": 1615422974, "cache_updated": false, "changed": false} 2021-03-11T00:39:19.629 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:19 +0000 (0:00:01.061) 0:02:56.248 ******** 2021-03-11T00:39:19.700 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:19.724 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:19 +0000 (0:00:00.094) 0:02:56.342 ******** 2021-03-11T00:39:19.749 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2021-03-11T00:39:19.751 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2021-03-11T00:39:19.784 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:19 +0000 (0:00:00.059) 0:02:56.402 ******** 2021-03-11T00:39:19.963 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2021-03-11T00:39:19.965 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2021-03-11T00:39:20.008 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:20 +0000 (0:00:00.224) 0:02:56.626 ******** 2021-03-11T00:39:20.407 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2021-03-11T00:39:20.410 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false} 2021-03-11T00:39:20.443 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:20 +0000 (0:00:00.434) 0:02:57.061 ******** 2021-03-11T00:39:20.514 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2021-03-11T00:39:20.516 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2021-03-11T00:39:20.566 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:20 +0000 (0:00:00.122) 0:02:57.184 ******** 2021-03-11T00:39:20.568 INFO:teuthology.task.ansible.out: 2021-03-11T00:39:20.627 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2021-03-11T00:39:20.629 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2021-03-11T00:39:20.674 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:20 +0000 (0:00:00.107) 0:02:57.292 ******** 2021-03-11T00:39:20.785 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2021-03-11T00:39:20.797 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1615422940.6629193, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1615422940.6589193, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 262873, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1615422940.6589193, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "3476867264", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-03-11T00:39:20.814 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:20 +0000 (0:00:00.139) 0:02:57.432 ******** 2021-03-11T00:39:21.498 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2021-03-11T00:39:21.500 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2021-03-11T00:39:21.528 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:21 +0000 (0:00:00.714) 0:02:58.147 ******** 2021-03-11T00:39:21.557 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:21 +0000 (0:00:00.028) 0:02:58.176 ******** 2021-03-11T00:39:21.580 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2021-03-11T00:39:21.582 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-03-11T00:39:21.622 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:21 +0000 (0:00:00.064) 0:02:58.240 ******** 2021-03-11T00:39:21.704 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2021-03-11T00:39:21.706 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:21.730 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:21 +0000 (0:00:00.107) 0:02:58.348 ******** 2021-03-11T00:39:21.950 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2021-03-11T00:39:21.952 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-03-11T00:39:21.979 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:21 +0000 (0:00:00.249) 0:02:58.597 ******** 2021-03-11T00:39:22.239 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2021-03-11T00:39:22.244 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-03-11T00:39:22.248 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2021-03-11T00:39:22.250 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false} 2021-03-11T00:39:22.316 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:22 +0000 (0:00:00.336) 0:02:58.934 ******** 2021-03-11T00:39:22.363 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:22 +0000 (0:00:00.047) 0:02:58.981 ******** 2021-03-11T00:39:22.374 INFO:teuthology.task.ansible.out: 2021-03-11T00:39:22.413 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2021-03-11T00:39:22.414 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:22.454 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:22 +0000 (0:00:00.089) 0:02:59.071 ******** 2021-03-11T00:39:22.474 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2021-03-11T00:39:22.476 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2021-03-11T00:39:22.524 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:22 +0000 (0:00:00.070) 0:02:59.142 ******** 2021-03-11T00:39:22.582 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2021-03-11T00:39:22.584 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2021-03-11T00:39:22.603 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2021-03-11T00:39:22.605 INFO:teuthology.task.ansible.out: skipping: [smithi119.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2021-03-11T00:39:22.650 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:22 +0000 (0:00:00.125) 0:02:59.268 ******** 2021-03-11T00:39:22.786 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2021-03-11T00:39:22.788 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi119.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:22.799 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:22.811 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:22.812 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2021-03-11T00:39:22.813 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1615422950.0027175, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1615422949.9987173, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 262873, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1615422949.9987173, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "3476867264", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-03-11T00:39:22.838 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:22 +0000 (0:00:00.188) 0:02:59.456 ******** 2021-03-11T00:39:22.869 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** skipping: [smithi119.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:22.924 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:22 +0000 (0:00:00.085) 0:02:59.542 ******** 2021-03-11T00:39:23.015 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-11T00:39:23.017 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:23.041 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:23 +0000 (0:00:00.116) 0:02:59.659 ******** 2021-03-11T00:39:23.100 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2021-03-11T00:39:23.102 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2021-03-11T00:39:23.144 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:23 +0000 (0:00:00.103) 0:02:59.763 ******** 2021-03-11T00:39:23.153 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2021-03-11T00:39:23.155 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:23.210 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:23 +0000 (0:00:00.065) 0:02:59.828 ******** 2021-03-11T00:39:23.231 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2021-03-11T00:39:23.233 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-03-11T00:39:23.259 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:23 +0000 (0:00:00.049) 0:02:59.878 ******** 2021-03-11T00:39:23.278 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-03-11T00:39:23.280 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:23.354 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:23 +0000 (0:00:00.094) 0:02:59.972 ******** 2021-03-11T00:39:23.381 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2021-03-11T00:39:23.389 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:23.476 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:23 +0000 (0:00:00.122) 0:03:00.095 ******** 2021-03-11T00:39:23.488 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2021-03-11T00:39:23.491 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:23.577 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:23 +0000 (0:00:00.100) 0:03:00.195 ******** 2021-03-11T00:39:23.661 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:23.717 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:23 +0000 (0:00:00.139) 0:03:00.335 ******** 2021-03-11T00:39:23.730 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2021-03-11T00:39:23.732 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-03-11T00:39:23.764 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:23 +0000 (0:00:00.047) 0:03:00.382 ******** 2021-03-11T00:39:23.784 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-03-11T00:39:23.786 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:23.843 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:23 +0000 (0:00:00.079) 0:03:00.461 ******** 2021-03-11T00:39:23.941 INFO:teuthology.task.ansible.out: 2021-03-11T00:39:23.943 INFO:teuthology.task.ansible.out:TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:23.974 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:23 +0000 (0:00:00.130) 0:03:00.592 ******** 2021-03-11T00:39:24.006 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2021-03-11T00:39:24.007 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-03-11T00:39:24.043 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:24 +0000 (0:00:00.069) 0:03:00.661 ******** 2021-03-11T00:39:24.045 INFO:teuthology.task.ansible.out: 2021-03-11T00:39:24.064 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-03-11T00:39:24.066 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:24.123 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:24 +0000 (0:00:00.080) 0:03:00.742 ******** 2021-03-11T00:39:24.143 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2021-03-11T00:39:24.144 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:24.188 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:24 +0000 (0:00:00.064) 0:03:00.806 ******** 2021-03-11T00:39:24.281 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2021-03-11T00:39:24.283 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2021-03-11T00:39:24.293 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2021-03-11T00:39:24.343 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2021-03-11T00:39:24.346 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2021-03-11T00:39:24.368 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2021-03-11T00:39:24.393 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:24 +0000 (0:00:00.204) 0:03:01.011 ******** 2021-03-11T00:39:24.436 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-03-11T00:39:24.439 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.011083", "end": "2021-03-11 00:39:24.390446", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-03-11 00:39:24.379363", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-11T00:39:24.468 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2021-03-11T00:39:24.470 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi124.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:24.470 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:24.471 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditiona 2021-03-11T00:39:24.473 INFO:teuthology.task.ansible.out:l result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:24.504 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:24 +0000 (0:00:00.111) 0:03:01.123 ******** 2021-03-11T00:39:24.506 INFO:teuthology.task.ansible.out: 2021-03-11T00:39:24.569 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-11T00:39:24.571 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:24.621 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:24 +0000 (0:00:00.116) 0:03:01.240 ******** 2021-03-11T00:39:24.631 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-03-11T00:39:24.632 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.006036", "end": "2021-03-11 00:39:24.565729", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-03-11 00:39:24.559693", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-11T00:39:24.687 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:24.718 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:24 +0000 (0:00:00.097) 0:03:01.337 ******** 2021-03-11T00:39:24.775 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-03-11T00:39:24.777 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005551", "end": "2021-03-11 00:39:24.748388", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-03-11 00:39:24.742837", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2021-03-11T00:39:24.789 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-03-11T00:39:24.791 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:24.820 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:24 +0000 (0:00:00.101) 0:03:01.438 ******** 2021-03-11T00:39:24.849 INFO:teuthology.task.ansible.out: 2021-03-11T00:39:24.851 INFO:teuthology.task.ansible.out:TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:24.881 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:24 +0000 (0:00:00.060) 0:03:01.499 ******** 2021-03-11T00:39:24.947 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-03-11T00:39:24.949 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.007907", "end": "2021-03-11 00:39:24.915702", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-03-11 00:39:24.907795", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-11T00:39:24.976 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:24 +0000 (0:00:00.095) 0:03:01.595 ******** 2021-03-11T00:39:24.988 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2021-03-11T00:39:24.992 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:25.056 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:25 +0000 (0:00:00.080) 0:03:01.675 ******** 2021-03-11T00:39:25.065 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-03-11T00:39:25.114 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:25 +0000 (0:00:00.057) 0:03:01.732 ******** 2021-03-11T00:39:25.140 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:25.191 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:25 +0000 (0:00:00.076) 0:03:01.809 ******** 2021-03-11T00:39:25.266 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:25.296 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:25 +0000 (0:00:00.105) 0:03:01.914 ******** 2021-03-11T00:39:25.308 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2021-03-11T00:39:25.310 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2021-03-11T00:39:25.379 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:25.404 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:25 +0000 (0:00:00.107) 0:03:02.022 ******** 2021-03-11T00:39:25.620 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-03-11T00:39:25.622 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.010988", "end": "2021-03-11 00:39:25.548271", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-03-11 00:39:25.537283", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-11T00:39:25.753 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.005366", "end": "2021-03-11 00:39:25.725479", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-03-11 00:39:25.720113", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-11T00:39:25.908 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005450", "end": "2021-03-11 00:39:25.885582", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-03-11 00:39:25.880132", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2021-03-11T00:39:26.165 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.091246", "end": "2021-03-11 00:39:26.116958", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-03-11 00:39:26.025712", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-11T00:39:26.191 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:26 +0000 (0:00:00.787) 0:03:02.809 ******** 2021-03-11T00:39:26.244 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-03-11T00:39:26.277 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:26 +0000 (0:00:00.086) 0:03:02.895 ******** 2021-03-11T00:39:26.466 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2021-03-11T00:39:26.467 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2021-03-11T00:39:26.554 INFO:teuthology.task.ansible.out: PLAY [testnodes,senta,vossi,jenkins_slaves] ************************************ 2021-03-11T00:39:26.582 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:26 +0000 (0:00:00.304) 0:03:03.200 ******** 2021-03-11T00:39:26.619 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:26 +0000 (0:00:00.035) 0:03:03.235 ******** 2021-03-11T00:39:27.442 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi119.front.sepia.ceph.com] 2021-03-11T00:39:27.483 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] 2021-03-11T00:39:27.529 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:27 +0000 (0:00:00.911) 0:03:04.147 ******** 2021-03-11T00:39:27.583 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-03-11T00:39:27.585 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2021-03-11T00:39:27.610 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:27 +0000 (0:00:00.081) 0:03:04.228 ******** 2021-03-11T00:39:27.654 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:27 +0000 (0:00:00.043) 0:03:04.272 ******** 2021-03-11T00:39:27.676 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-03-11T00:39:27.678 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:27.702 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:27 +0000 (0:00:00.048) 0:03:04.321 ******** 2021-03-11T00:39:27.714 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-03-11T00:39:27.716 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2021-03-11T00:39:27.751 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:27 +0000 (0:00:00.048) 0:03:04.369 ******** 2021-03-11T00:39:27.795 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:27.819 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:27 +0000 (0:00:00.067) 0:03:04.437 ******** 2021-03-11T00:39:27.831 INFO:teuthology.task.ansible.out: TASK [container-host : Including distro specific variables] ******************** ok: [smithi119.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2021-03-11T00:39:27.874 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:27 +0000 (0:00:00.055) 0:03:04.492 ******** 2021-03-11T00:39:27.894 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2021-03-11T00:39:27.928 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:27 +0000 (0:00:00.053) 0:03:04.546 ******** 2021-03-11T00:39:28.819 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** 2021-03-11T00:39:28.822 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"cache_update_time": 1615422969, "cache_updated": false, "changed": false} 2021-03-11T00:39:28.849 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:28 +0000 (0:00:00.921) 0:03:05.468 ******** 2021-03-11T00:39:28.851 INFO:teuthology.task.ansible.out: 2021-03-11T00:39:28.879 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-03-11T00:39:28.881 INFO:teuthology.task.ansible.out:skipping: [smithi119.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:28.923 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:28 +0000 (0:00:00.073) 0:03:05.541 ******** 2021-03-11T00:39:28.924 INFO:teuthology.task.ansible.out: 2021-03-11T00:39:28.959 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** 2021-03-11T00:39:28.961 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"cache_update_time": 1615422974, "cache_updated": false, "changed": false} 2021-03-11T00:39:29.018 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:29 +0000 (0:00:00.094) 0:03:05.636 ******** 2021-03-11T00:39:29.045 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-03-11T00:39:29.047 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2021-03-11T00:39:29.097 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:29 +0000 (0:00:00.079) 0:03:05.715 ******** 2021-03-11T00:39:29.114 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-03-11T00:39:29.117 INFO:teuthology.task.ansible.out:skipping: [smithi124.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-11T00:39:29.142 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:29 +0000 (0:00:00.045) 0:03:05.761 ******** 2021-03-11T00:39:29.207 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** ok: [smithi124.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2021-03-11T00:39:29.228 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:29 +0000 (0:00:00.085) 0:03:05.846 ******** 2021-03-11T00:39:29.310 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-03-11T00:39:29.312 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-03-11T00:39:29.455 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-03-11T00:39:29.457 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-03-11T00:39:29.485 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:29 +0000 (0:00:00.255) 0:03:06.102 ******** 2021-03-11T00:39:29.618 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-03-11T00:39:29.668 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:29 +0000 (0:00:00.175) 0:03:06.278 ******** 2021-03-11T00:39:30.119 INFO:teuthology.task.ansible.out: 2021-03-11T00:39:30.121 INFO:teuthology.task.ansible.out:TASK [container-host : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** changed: [smithi119.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615423169.5756245-21124-110143134070445/source", "state": "file", "uid": 0} 2021-03-11T00:39:30.206 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615423169.7554498-21164-241530721920775/source", "state": "file", "uid": 0} 2021-03-11T00:39:30.776 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615423170.1805935-21124-214056761779544/source", "state": "file", "uid": 0} 2021-03-11T00:39:30.804 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:30 +0000 (0:00:01.144) 0:03:07.422 ******** 2021-03-11T00:39:30.813 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615423170.2692084-21164-141667733256622/source", "state": "file", "uid": 0} 2021-03-11T00:39:30.871 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:30 +0000 (0:00:00.067) 0:03:07.489 ******** 2021-03-11T00:39:34.186 INFO:teuthology.task.ansible.out: TASK [container-host : Install registries-conf-ctl] **************************** 2021-03-11T00:39:34.188 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/bin/pip2", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": "", "stderr_lines": [], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-Ap_zg5-build\nCollecting docopt (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\nCollecting toml (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl\nCollecting typing (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl\nBuilding wheels for collected packages: docopt\n Running setup.py bdist_wheel for docopt: started\n Running setup.py bdist_wheel for docopt: finished with status 'done'\n Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e\nSuccessfully built docopt\nInstalling collected packages: docopt, toml, typing, registries-conf-ctl\n Running setup.py install for registries-conf-ctl: started\n Running setup.py install for registries-conf-ctl: finished with status 'done'\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-Ap_zg5-build", "Collecting docopt (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packag 2021-03-11T00:39:34.189 INFO:teuthology.task.ansible.out:es/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz", "Collecting toml (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl", "Collecting typing (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl", "Building wheels for collected packages: docopt", " Running setup.py bdist_wheel for docopt: started", " Running setup.py bdist_wheel for docopt: finished with status 'done'", " Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e", "Successfully built docopt", "Installing collected packages: docopt, toml, typing, registries-conf-ctl", " Running setup.py install for registries-conf-ctl: started", " Running setup.py install for registries-conf-ctl: finished with status 'done'", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3"], "version": null, "virtualenv": null} 2021-03-11T00:39:34.220 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:34 +0000 (0:00:03.349) 0:03:10.838 ******** 2021-03-11T00:39:34.233 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/bin/pip2", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": "", "stderr_lines": [], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-ruX_88-build\nCollecting docopt (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\nCollecting toml (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl\nCollecting typing (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl\nBuilding wheels for collected packages: docopt\n Running setup.py bdist_wheel for docopt: started\n Running setup.py bdist_wheel for docopt: finished with status 'done'\n Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e\nSuccessfully built docopt\nInstalling collected packages: docopt, toml, typing, registries-conf-ctl\n Running setup.py install for registries-conf-ctl: started\n Running setup.py install for registries-conf-ctl: finished with status 'done'\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-ruX_88-build", "Collecting docopt (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packag 2021-03-11T00:39:34.235 INFO:teuthology.task.ansible.out:es/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz", "Collecting toml (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl", "Collecting typing (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl", "Building wheels for collected packages: docopt", " Running setup.py bdist_wheel for docopt: started", " Running setup.py bdist_wheel for docopt: finished with status 'done'", " Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e", "Successfully built docopt", "Installing collected packages: docopt, toml, typing, registries-conf-ctl", " Running setup.py install for registries-conf-ctl: started", " Running setup.py install for registries-conf-ctl: finished with status 'done'", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3"], "version": null, "virtualenv": null} 2021-03-11T00:39:34.269 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:34 +0000 (0:00:00.049) 0:03:10.888 ******** 2021-03-11T00:39:34.456 INFO:teuthology.task.ansible.out: TASK [container-host : Check for docker's daemon.json] ************************* 2021-03-11T00:39:34.458 INFO:teuthology.task.ansible.out:ok: [smithi119.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2021-03-11T00:39:34.494 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:34 +0000 (0:00:00.223) 0:03:11.111 ******** 2021-03-11T00:39:34.510 INFO:teuthology.task.ansible.out:ok: [smithi124.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2021-03-11T00:39:34.555 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:34 +0000 (0:00:00.062) 0:03:11.173 ******** 2021-03-11T00:39:35.031 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/daemon.json if necessary] ************ 2021-03-11T00:39:35.033 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => {"changed": true, "checksum": "bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f", "dest": "/etc/docker/daemon.json", "gid": 0, "group": "root", "md5sum": "99914b932bd37a50b983c5e7c90ae93b", "mode": "0644", "owner": "root", "size": 2, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615423174.601105-22292-56659184838832/source", "state": "file", "uid": 0} 2021-03-11T00:39:35.057 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:35 +0000 (0:00:00.502) 0:03:11.676 ******** 2021-03-11T00:39:35.163 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => {"changed": true, "checksum": "bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f", "dest": "/etc/docker/daemon.json", "gid": 0, "group": "root", "md5sum": "99914b932bd37a50b983c5e7c90ae93b", "mode": "0644", "owner": "root", "size": 2, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615423174.6469107-22304-98893303005649/source", "state": "file", "uid": 0} 2021-03-11T00:39:35.190 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:35 +0000 (0:00:00.132) 0:03:11.809 ******** 2021-03-11T00:39:35.357 INFO:teuthology.task.ansible.out: TASK [container-host : Add local docker.io registry mirror] ******************** 2021-03-11T00:39:35.359 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.127349", "end": "2021-03-11 00:39:35.335701", "rc": 0, "start": "2021-03-11 00:39:35.208352", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-11T00:39:35.386 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:35 +0000 (0:00:00.195) 0:03:12.004 ******** 2021-03-11T00:39:35.561 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.203496", "end": "2021-03-11 00:39:35.530487", "rc": 0, "start": "2021-03-11 00:39:35.326991", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-11T00:39:35.590 INFO:teuthology.task.ansible.out:Thursday 11 March 2021 00:39:35 +0000 (0:00:00.204) 0:03:12.209 ******** 2021-03-11T00:39:38.699 INFO:teuthology.task.ansible.out: TASK [container-host : Restart docker service] ********************************* 2021-03-11T00:39:38.700 INFO:teuthology.task.ansible.out:changed: [smithi119.front.sepia.ceph.com] => {"changed": true, "name": "docker", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "system.slice sysinit.target docker.socket network-online.target systemd-journald.socket basic.target containerd.service firewalld.service", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ConsistsOf": "docker.socket", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "yes", "DelegateControllers": "cpu cpuacct io blkio memory devices pids", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "https://docs.docker.com", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMa 2021-03-11T00:39:38.702 INFO:teuthology.task.ansible.out:inStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/docker.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "process", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "infinity", "LimitNOFILESoft": "infinity", "LimitNPROC": "infinity", "LimitNPROCSoft": "infinity", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "12802 2021-03-11T00:39:38.704 INFO:teuthology.task.ansible.out:9", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target docker.socket", "Restart": "always", "RestartUSec": "2s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitIntervalUSec": "1min", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Thu 2021-03-11 00:35:43 UTC", "StateChangeTimestampMonotonic": "40667336", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "Su 2021-03-11T00:39:38.705 INFO:teuthology.task.ansible.out:ccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "infinity", "TimeoutStartUSec": "infinity", "TimeoutStopUSec": "infinity", "TimerSlackNSec": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "notify", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-online.target containerd.service", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-03-11T00:39:38.980 INFO:teuthology.task.ansible.out:changed: [smithi124.front.sepia.ceph.com] => {"changed": true, "name": "docker", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "systemd-journald.socket network-online.target system.slice sysinit.target basic.target firewalld.service containerd.service docker.socket", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ConsistsOf": "docker.socket", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "yes", "DelegateControllers": "cpu cpuacct io blkio memory devices pids", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "https://docs.docker.com", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMa 2021-03-11T00:39:38.981 INFO:teuthology.task.ansible.out:inStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/docker.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "process", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "infinity", "LimitNOFILESoft": "infinity", "LimitNPROC": "infinity", "LimitNPROCSoft": "infinity", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128028", "LimitSIGPENDINGSoft": "12802 2021-03-11T00:39:38.983 INFO:teuthology.task.ansible.out:8", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "docker.socket system.slice sysinit.target", "Restart": "always", "RestartUSec": "2s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitIntervalUSec": "1min", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Thu 2021-03-11 00:35:52 UTC", "StateChangeTimestampMonotonic": "41088631", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "Su 2021-03-11T00:39:38.985 INFO:teuthology.task.ansible.out:ccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "infinity", "TimeoutStartUSec": "infinity", "TimeoutStopUSec": "infinity", "TimerSlackNSec": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "notify", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-online.target containerd.service", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-03-11T00:39:39.064 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2021-03-11T00:39:39.066 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2021-03-11T00:39:39.067 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched 2021-03-11T00:39:39.067 INFO:teuthology.task.ansible.out: PLAY RECAP ********************************************************************* smithi119.front.sepia.ceph.com : ok=127 changed=35 unreachable=0 failed=0 skipped=184 rescued=0 ignored=0 smithi124.front.sepia.ceph.com : ok=127 changed=35 unreachable=0 failed=0 skipped=184 rescued=0 ignored=0 Thursday 11 March 2021 00:39:39 +0000 (0:00:03.476) 0:03:15.685 ******** =============================================================================== 2021-03-11T00:39:39.068 INFO:teuthology.task.ansible.out:Create all admin users with sudo access. ------------------------------- 21.80s users : Update authorized_keys using the keys repo --------------------- 20.43s testnode : Install packages -------------------------------------------- 10.42s testnode : Ensure packages are not present. ----------------------------- 9.95s Remove revoked users ---------------------------------------------------- 7.20s testnode : Zap all non-root disks --------------------------------------- 6.10s container-host : Restart docker service --------------------------------- 3.48s container-host : Install registries-conf-ctl ---------------------------- 3.35s common : Update apt cache ----------------------------------------------- 3.13s testnode : Update apt cache. -------------------------------------------- 2.75s testnode : Install apt keys --------------------------------------------- 2.12s users : Update authorized_keys for each user with literal keys ---------- 2.12s Gathering Facts --------------------------------------------------------- 1.79s testnode : Make sure apt dependencies are installed --------------------- 1.22s common : Install nrpe package and dependencies (Ubuntu/Fedora) ---------- 1.15s container-host : Copy docker-mirror.front.sepia.ceph.com:5000 sel 2021-03-11T00:39:39.070 INFO:teuthology.task.ansible.out:f-signed cert --- 1.14s testnode : Set the system's hostname ------------------------------------ 1.08s testnode : Install python-pip on apt based systems. --------------------- 1.05s common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/. --- 1.02s testnode : Upgrade packages --------------------------------------------- 1.00s 2021-03-11T00:39:39.634 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi119.front.sepia.ceph.com'), Remote(name='ubuntu@smithi124.front.sepia.ceph.com')] 2021-03-11T00:39:39.636 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-11T00:39:39.637 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-03-11T00:39:39.763 DEBUG:teuthology.orchestra.run.smithi119:> true 2021-03-11T00:39:39.959 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-11T00:39:39.960 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-03-11T00:39:40.036 DEBUG:teuthology.orchestra.run.smithi124:> true 2021-03-11T00:39:40.240 INFO:teuthology.run_tasks:Running task clock... 2021-03-11T00:39:40.254 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2021-03-11T00:39:40.255 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-03-11T00:39:40.257 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-03-11T00:39:40.316 INFO:teuthology.orchestra.run.smithi119.stdout:11 Mar 00:39:40 ntpd[9339]: ntpd 4.2.8p10@1.3728-o (1): Starting 2021-03-11T00:39:40.317 INFO:teuthology.orchestra.run.smithi119.stdout:11 Mar 00:39:40 ntpd[9339]: Command line: ntpd -gq 2021-03-11T00:39:40.317 INFO:teuthology.orchestra.run.smithi119.stdout:11 Mar 00:39:40 ntpd[9339]: proto: precision = 0.123 usec (-23) 2021-03-11T00:39:40.318 INFO:teuthology.orchestra.run.smithi119.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-03-11T00:39:40.319 INFO:teuthology.orchestra.run.smithi119.stdout:11 Mar 00:39:40 ntpd[9339]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-03-11T00:39:40.319 INFO:teuthology.orchestra.run.smithi119.stderr:restrict ::: KOD does nothing without LIMITED. 2021-03-11T00:39:40.321 INFO:teuthology.orchestra.run.smithi119.stdout:11 Mar 00:39:40 ntpd[9339]: restrict ::: KOD does nothing without LIMITED. 2021-03-11T00:39:40.321 INFO:teuthology.orchestra.run.smithi119.stdout:11 Mar 00:39:40 ntpd[9339]: Listen and drop on 0 v6wildcard [::]:123 2021-03-11T00:39:40.322 INFO:teuthology.orchestra.run.smithi119.stdout:11 Mar 00:39:40 ntpd[9339]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2021-03-11T00:39:40.322 INFO:teuthology.orchestra.run.smithi119.stdout:11 Mar 00:39:40 ntpd[9339]: Listen normally on 2 lo 127.0.0.1:123 2021-03-11T00:39:40.323 INFO:teuthology.orchestra.run.smithi119.stdout:11 Mar 00:39:40 ntpd[9339]: Listen normally on 3 enp3s0f1 172.21.15.119:123 2021-03-11T00:39:40.323 INFO:teuthology.orchestra.run.smithi119.stdout:11 Mar 00:39:40 ntpd[9339]: Listen normally on 4 lo [::1]:123 2021-03-11T00:39:40.323 INFO:teuthology.orchestra.run.smithi119.stdout:11 Mar 00:39:40 ntpd[9339]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d5ef%5]:123 2021-03-11T00:39:40.323 INFO:teuthology.orchestra.run.smithi119.stdout:11 Mar 00:39:40 ntpd[9339]: Listening on routing socket on fd #22 for interface updates 2021-03-11T00:39:41.317 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:41 ntpd[9339]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-11T00:39:41.317 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:41 ntpd[9339]: can't open /var/log/ntpstats/rawstats.20210311: Permission denied 2021-03-11T00:39:41.318 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:41 ntpd[9339]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-11T00:39:41.318 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:41 ntpd[9339]: can't open /var/log/ntpstats/peerstats.20210311: Permission denied 2021-03-11T00:39:41.318 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:41 ntpd[9339]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-11T00:39:41.318 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:41 ntpd[9339]: can't open /var/log/ntpstats/rawstats.20210311: Permission denied 2021-03-11T00:39:41.319 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:41 ntpd[9339]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-11T00:39:41.319 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:41 ntpd[9339]: can't open /var/log/ntpstats/peerstats.20210311: Permission denied 2021-03-11T00:39:41.319 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:41 ntpd[9339]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-11T00:39:41.319 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:41 ntpd[9339]: can't open /var/log/ntpstats/rawstats.20210311: Permission denied 2021-03-11T00:39:41.319 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:41 ntpd[9339]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-11T00:39:41.320 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:41 ntpd[9339]: can't open /var/log/ntpstats/peerstats.20210311: Permission denied 2021-03-11T00:39:43.318 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:43 ntpd[9339]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-11T00:39:43.318 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:43 ntpd[9339]: can't open /var/log/ntpstats/rawstats.20210311: Permission denied 2021-03-11T00:39:43.318 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:43 ntpd[9339]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-11T00:39:43.319 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:43 ntpd[9339]: can't open /var/log/ntpstats/peerstats.20210311: Permission denied 2021-03-11T00:39:43.319 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:43 ntpd[9339]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-11T00:39:43.319 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:43 ntpd[9339]: can't open /var/log/ntpstats/rawstats.20210311: Permission denied 2021-03-11T00:39:43.319 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:43 ntpd[9339]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-11T00:39:43.319 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:43 ntpd[9339]: can't open /var/log/ntpstats/peerstats.20210311: Permission denied 2021-03-11T00:39:43.320 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:43 ntpd[9339]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-11T00:39:43.320 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:43 ntpd[9339]: can't open /var/log/ntpstats/rawstats.20210311: Permission denied 2021-03-11T00:39:43.320 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:43 ntpd[9339]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-11T00:39:43.320 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:43 ntpd[9339]: can't open /var/log/ntpstats/peerstats.20210311: Permission denied 2021-03-11T00:39:43.322 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:43 ntpd[9339]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-11T00:39:43.323 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:43 ntpd[9339]: can't open /var/log/ntpstats/rawstats.20210311: Permission denied 2021-03-11T00:39:43.323 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:43 ntpd[9339]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-11T00:39:43.323 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:43 ntpd[9339]: can't open /var/log/ntpstats/peerstats.20210311: Permission denied 2021-03-11T00:39:45.320 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:45 ntpd[9339]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-11T00:39:45.321 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:45 ntpd[9339]: can't open /var/log/ntpstats/rawstats.20210311: Permission denied 2021-03-11T00:39:45.321 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:45 ntpd[9339]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-11T00:39:45.321 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:45 ntpd[9339]: can't open /var/log/ntpstats/peerstats.20210311: Permission denied 2021-03-11T00:39:45.321 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:45 ntpd[9339]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-11T00:39:45.322 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:45 ntpd[9339]: can't open /var/log/ntpstats/rawstats.20210311: Permission denied 2021-03-11T00:39:45.322 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:45 ntpd[9339]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-11T00:39:45.322 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:45 ntpd[9339]: can't open /var/log/ntpstats/peerstats.20210311: Permission denied 2021-03-11T00:39:45.322 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:45 ntpd[9339]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-11T00:39:45.322 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:45 ntpd[9339]: can't open /var/log/ntpstats/rawstats.20210311: Permission denied 2021-03-11T00:39:45.323 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:45 ntpd[9339]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-11T00:39:45.323 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:45 ntpd[9339]: can't open /var/log/ntpstats/peerstats.20210311: Permission denied 2021-03-11T00:39:45.324 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:45 ntpd[9339]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-11T00:39:45.325 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:45 ntpd[9339]: can't open /var/log/ntpstats/rawstats.20210311: Permission denied 2021-03-11T00:39:45.325 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:45 ntpd[9339]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-11T00:39:45.325 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:45 ntpd[9339]: can't open /var/log/ntpstats/peerstats.20210311: Permission denied 2021-03-11T00:39:47.317 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:47 ntpd[9339]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-11T00:39:47.317 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:47 ntpd[9339]: can't open /var/log/ntpstats/rawstats.20210311: Permission denied 2021-03-11T00:39:47.317 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:47 ntpd[9339]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-11T00:39:47.318 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:47 ntpd[9339]: can't open /var/log/ntpstats/peerstats.20210311: Permission denied 2021-03-11T00:39:47.318 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:47 ntpd[9339]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-11T00:39:47.318 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:47 ntpd[9339]: can't open /var/log/ntpstats/rawstats.20210311: Permission denied 2021-03-11T00:39:47.318 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:47 ntpd[9339]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-11T00:39:47.319 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:47 ntpd[9339]: can't open /var/log/ntpstats/peerstats.20210311: Permission denied 2021-03-11T00:39:47.319 INFO:teuthology.orchestra.run.smithi119.stdout:11 Mar 00:39:47 ntpd[9339]: ntpd: time slew -0.000776 s 2021-03-11T00:39:47.319 INFO:teuthology.orchestra.run.smithi119.stdout:ntpd: time slew -0.000776s 2021-03-11T00:39:47.320 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:47 ntpd[9339]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2021-03-11T00:39:47.321 INFO:teuthology.orchestra.run.smithi119.stderr:11 Mar 00:39:47 ntpd[9339]: can't open /var/log/ntpstats/loopstats.20210311: Permission denied 2021-03-11T00:39:47.371 INFO:teuthology.orchestra.run.smithi119.stdout: remote refid st t when poll reach delay offset jitter 2021-03-11T00:39:47.372 INFO:teuthology.orchestra.run.smithi119.stdout:============================================================================== 2021-03-11T00:39:47.372 INFO:teuthology.orchestra.run.smithi119.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-03-11T00:39:47.372 INFO:teuthology.orchestra.run.smithi119.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-03-11T00:39:47.372 INFO:teuthology.orchestra.run.smithi119.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-03-11T00:39:47.373 INFO:teuthology.orchestra.run.smithi119.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-03-11T00:39:47.374 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-03-11T00:39:47.375 DEBUG:teuthology.orchestra.run.smithi124:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-03-11T00:39:47.441 INFO:teuthology.orchestra.run.smithi124.stdout:11 Mar 00:39:47 ntpd[9322]: ntpd 4.2.8p10@1.3728-o (1): Starting 2021-03-11T00:39:47.441 INFO:teuthology.orchestra.run.smithi124.stdout:11 Mar 00:39:47 ntpd[9322]: Command line: ntpd -gq 2021-03-11T00:39:47.442 INFO:teuthology.orchestra.run.smithi124.stdout:11 Mar 00:39:47 ntpd[9322]: proto: precision = 0.123 usec (-23) 2021-03-11T00:39:47.442 INFO:teuthology.orchestra.run.smithi124.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-03-11T00:39:47.443 INFO:teuthology.orchestra.run.smithi124.stdout:11 Mar 00:39:47 ntpd[9322]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-03-11T00:39:47.443 INFO:teuthology.orchestra.run.smithi124.stderr:restrict ::: KOD does nothing without LIMITED. 2021-03-11T00:39:47.445 INFO:teuthology.orchestra.run.smithi124.stdout:11 Mar 00:39:47 ntpd[9322]: restrict ::: KOD does nothing without LIMITED. 2021-03-11T00:39:47.445 INFO:teuthology.orchestra.run.smithi124.stdout:11 Mar 00:39:47 ntpd[9322]: Listen and drop on 0 v6wildcard [::]:123 2021-03-11T00:39:47.445 INFO:teuthology.orchestra.run.smithi124.stdout:11 Mar 00:39:47 ntpd[9322]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2021-03-11T00:39:47.445 INFO:teuthology.orchestra.run.smithi124.stdout:11 Mar 00:39:47 ntpd[9322]: Listen normally on 2 lo 127.0.0.1:123 2021-03-11T00:39:47.446 INFO:teuthology.orchestra.run.smithi124.stdout:11 Mar 00:39:47 ntpd[9322]: Listen normally on 3 enp3s0f1 172.21.15.124:123 2021-03-11T00:39:47.446 INFO:teuthology.orchestra.run.smithi124.stdout:11 Mar 00:39:47 ntpd[9322]: Listen normally on 4 lo [::1]:123 2021-03-11T00:39:47.446 INFO:teuthology.orchestra.run.smithi124.stdout:11 Mar 00:39:47 ntpd[9322]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d5c5%5]:123 2021-03-11T00:39:47.446 INFO:teuthology.orchestra.run.smithi124.stdout:11 Mar 00:39:47 ntpd[9322]: Listening on routing socket on fd #22 for interface updates 2021-03-11T00:39:48.442 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:48 ntpd[9322]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-11T00:39:48.443 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:48 ntpd[9322]: can't open /var/log/ntpstats/rawstats.20210311: Permission denied 2021-03-11T00:39:48.443 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:48 ntpd[9322]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-11T00:39:48.443 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:48 ntpd[9322]: can't open /var/log/ntpstats/peerstats.20210311: Permission denied 2021-03-11T00:39:48.443 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:48 ntpd[9322]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-11T00:39:48.443 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:48 ntpd[9322]: can't open /var/log/ntpstats/rawstats.20210311: Permission denied 2021-03-11T00:39:48.444 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:48 ntpd[9322]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-11T00:39:48.444 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:48 ntpd[9322]: can't open /var/log/ntpstats/peerstats.20210311: Permission denied 2021-03-11T00:39:49.441 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:49 ntpd[9322]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-11T00:39:49.442 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:49 ntpd[9322]: can't open /var/log/ntpstats/rawstats.20210311: Permission denied 2021-03-11T00:39:49.442 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:49 ntpd[9322]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-11T00:39:49.442 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:49 ntpd[9322]: can't open /var/log/ntpstats/peerstats.20210311: Permission denied 2021-03-11T00:39:50.443 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:50 ntpd[9322]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-11T00:39:50.443 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:50 ntpd[9322]: can't open /var/log/ntpstats/rawstats.20210311: Permission denied 2021-03-11T00:39:50.444 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:50 ntpd[9322]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-11T00:39:50.444 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:50 ntpd[9322]: can't open /var/log/ntpstats/peerstats.20210311: Permission denied 2021-03-11T00:39:50.444 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:50 ntpd[9322]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-11T00:39:50.445 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:50 ntpd[9322]: can't open /var/log/ntpstats/rawstats.20210311: Permission denied 2021-03-11T00:39:50.445 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:50 ntpd[9322]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-11T00:39:50.445 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:50 ntpd[9322]: can't open /var/log/ntpstats/peerstats.20210311: Permission denied 2021-03-11T00:39:51.442 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:51 ntpd[9322]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-11T00:39:51.442 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:51 ntpd[9322]: can't open /var/log/ntpstats/rawstats.20210311: Permission denied 2021-03-11T00:39:51.443 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:51 ntpd[9322]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-11T00:39:51.443 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:51 ntpd[9322]: can't open /var/log/ntpstats/peerstats.20210311: Permission denied 2021-03-11T00:39:51.443 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:51 ntpd[9322]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-11T00:39:51.443 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:51 ntpd[9322]: can't open /var/log/ntpstats/rawstats.20210311: Permission denied 2021-03-11T00:39:51.444 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:51 ntpd[9322]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-11T00:39:51.444 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:51 ntpd[9322]: can't open /var/log/ntpstats/peerstats.20210311: Permission denied 2021-03-11T00:39:52.442 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:52 ntpd[9322]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-11T00:39:52.442 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:52 ntpd[9322]: can't open /var/log/ntpstats/rawstats.20210311: Permission denied 2021-03-11T00:39:52.443 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:52 ntpd[9322]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-11T00:39:52.443 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:52 ntpd[9322]: can't open /var/log/ntpstats/peerstats.20210311: Permission denied 2021-03-11T00:39:52.443 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:52 ntpd[9322]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-11T00:39:52.443 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:52 ntpd[9322]: can't open /var/log/ntpstats/rawstats.20210311: Permission denied 2021-03-11T00:39:52.444 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:52 ntpd[9322]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-11T00:39:52.444 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:52 ntpd[9322]: can't open /var/log/ntpstats/peerstats.20210311: Permission denied 2021-03-11T00:39:53.442 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:53 ntpd[9322]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-11T00:39:53.442 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:53 ntpd[9322]: can't open /var/log/ntpstats/rawstats.20210311: Permission denied 2021-03-11T00:39:53.443 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:53 ntpd[9322]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-11T00:39:53.443 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:53 ntpd[9322]: can't open /var/log/ntpstats/peerstats.20210311: Permission denied 2021-03-11T00:39:53.443 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:53 ntpd[9322]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-11T00:39:53.443 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:53 ntpd[9322]: can't open /var/log/ntpstats/rawstats.20210311: Permission denied 2021-03-11T00:39:53.444 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:53 ntpd[9322]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-11T00:39:53.444 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:53 ntpd[9322]: can't open /var/log/ntpstats/peerstats.20210311: Permission denied 2021-03-11T00:39:54.441 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:54 ntpd[9322]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-11T00:39:54.442 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:54 ntpd[9322]: can't open /var/log/ntpstats/rawstats.20210311: Permission denied 2021-03-11T00:39:54.442 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:54 ntpd[9322]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-11T00:39:54.442 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:54 ntpd[9322]: can't open /var/log/ntpstats/peerstats.20210311: Permission denied 2021-03-11T00:39:54.443 INFO:teuthology.orchestra.run.smithi124.stdout:11 Mar 00:39:54 ntpd[9322]: ntpd: time slew -0.002432 s 2021-03-11T00:39:54.443 INFO:teuthology.orchestra.run.smithi124.stdout:ntpd: time slew -0.002432s 2021-03-11T00:39:54.444 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:54 ntpd[9322]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2021-03-11T00:39:54.444 INFO:teuthology.orchestra.run.smithi124.stderr:11 Mar 00:39:54 ntpd[9322]: can't open /var/log/ntpstats/loopstats.20210311: Permission denied 2021-03-11T00:39:54.499 INFO:teuthology.orchestra.run.smithi124.stdout: remote refid st t when poll reach delay offset jitter 2021-03-11T00:39:54.500 INFO:teuthology.orchestra.run.smithi124.stdout:============================================================================== 2021-03-11T00:39:54.500 INFO:teuthology.orchestra.run.smithi124.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-03-11T00:39:54.500 INFO:teuthology.orchestra.run.smithi124.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-03-11T00:39:54.501 INFO:teuthology.orchestra.run.smithi124.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-03-11T00:39:54.501 INFO:teuthology.orchestra.run.smithi124.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-03-11T00:39:54.502 INFO:teuthology.run_tasks:Running task install... 2021-03-11T00:39:54.514 DEBUG:teuthology.task.install:project ceph 2021-03-11T00:39:54.515 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'branch': 'octopus', 'sha1': '409dc4b843da18e60e08a06ad16195db022fb227'}} 2021-03-11T00:39:54.516 DEBUG:teuthology.task.install:config {'branch': 'octopus', 'sha1': '409dc4b843da18e60e08a06ad16195db022fb227'} 2021-03-11T00:39:54.516 INFO:teuthology.task.install:Using flavor: basic 2021-03-11T00:39:54.546 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2021-03-11T00:39:54.553 INFO:teuthology.task.install:extra packages: [] 2021-03-11T00:39:54.555 DEBUG:teuthology.orchestra.run.smithi119:> sudo lsb_release -is 2021-03-11T00:39:54.601 INFO:teuthology.orchestra.run.smithi119.stdout:Ubuntu 2021-03-11T00:39:54.607 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-03-11T00:39:54.608 DEBUG:teuthology.orchestra.run.smithi124:> sudo lsb_release -is 2021-03-11T00:39:54.610 DEBUG:teuthology.orchestra.run.smithi119:> sudo apt-key list | grep Ceph 2021-03-11T00:39:54.663 INFO:teuthology.orchestra.run.smithi124.stdout:Ubuntu 2021-03-11T00:39:54.669 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-03-11T00:39:54.670 DEBUG:teuthology.orchestra.run.smithi124:> sudo apt-key list | grep Ceph 2021-03-11T00:39:54.700 INFO:teuthology.orchestra.run.smithi119.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2021-03-11T00:39:54.757 INFO:teuthology.orchestra.run.smithi119.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2021-03-11T00:39:54.758 INFO:teuthology.orchestra.run.smithi119.stdout:uid [ unknown] Ceph.com (release key) 2021-03-11T00:39:54.759 DEBUG:teuthology.orchestra.run.smithi119:> sudo lsb_release -is 2021-03-11T00:39:54.765 INFO:teuthology.orchestra.run.smithi124.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2021-03-11T00:39:54.812 INFO:teuthology.orchestra.run.smithi119.stdout:Ubuntu 2021-03-11T00:39:54.817 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-03-11T00:39:54.818 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2021-03-11T00:39:54.819 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2021-03-11T00:39:54.819 INFO:teuthology.packaging:ref: None 2021-03-11T00:39:54.820 INFO:teuthology.packaging:tag: None 2021-03-11T00:39:54.821 INFO:teuthology.packaging:branch: octopus 2021-03-11T00:39:54.821 INFO:teuthology.packaging:sha1: 409dc4b843da18e60e08a06ad16195db022fb227 2021-03-11T00:39:54.822 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=octopus 2021-03-11T00:39:54.836 INFO:teuthology.orchestra.run.smithi124.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2021-03-11T00:39:54.837 INFO:teuthology.orchestra.run.smithi124.stdout:uid [ unknown] Ceph.com (release key) 2021-03-11T00:39:54.838 DEBUG:teuthology.orchestra.run.smithi124:> sudo lsb_release -is 2021-03-11T00:39:54.897 INFO:teuthology.orchestra.run.smithi124.stdout:Ubuntu 2021-03-11T00:39:54.903 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-03-11T00:39:54.904 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2021-03-11T00:39:54.904 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2021-03-11T00:39:54.905 INFO:teuthology.packaging:ref: None 2021-03-11T00:39:54.906 INFO:teuthology.packaging:tag: None 2021-03-11T00:39:54.906 INFO:teuthology.packaging:branch: octopus 2021-03-11T00:39:54.907 INFO:teuthology.packaging:sha1: 409dc4b843da18e60e08a06ad16195db022fb227 2021-03-11T00:39:54.907 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=octopus 2021-03-11T00:39:55.063 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default/ 2021-03-11T00:39:55.064 INFO:teuthology.task.install.deb:Package version is 15.2.9-108-g1ff7d244-1bionic 2021-03-11T00:39:55.128 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default/ 2021-03-11T00:39:55.129 INFO:teuthology.task.install.deb:Package version is 15.2.9-108-g1ff7d244-1bionic 2021-03-11T00:39:55.194 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2021-03-11T00:39:55.197 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2021-03-11T00:39:55.272 DEBUG:teuthology.orchestra.run.smithi119:> sudo apt-get update 2021-03-11T00:39:55.274 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2021-03-11T00:39:55.274 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2021-03-11T00:39:55.345 DEBUG:teuthology.orchestra.run.smithi124:> sudo apt-get update 2021-03-11T00:39:55.510 INFO:teuthology.orchestra.run.smithi119.stdout:Ign:1 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic InRelease 2021-03-11T00:39:55.531 INFO:teuthology.orchestra.run.smithi119.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2021-03-11T00:39:55.553 INFO:teuthology.orchestra.run.smithi119.stdout:Ign:3 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic Release.gpg 2021-03-11T00:39:55.582 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:4 http://security.ubuntu.com/ubuntu bionic-security InRelease 2021-03-11T00:39:55.583 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:5 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-03-11T00:39:55.583 INFO:teuthology.orchestra.run.smithi119.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,085 B] 2021-03-11T00:39:55.600 INFO:teuthology.orchestra.run.smithi124.stdout:Ign:1 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic InRelease 2021-03-11T00:39:55.612 INFO:teuthology.orchestra.run.smithi119.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 Packages [15.8 kB] 2021-03-11T00:39:55.614 INFO:teuthology.orchestra.run.smithi124.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2021-03-11T00:39:55.637 INFO:teuthology.orchestra.run.smithi124.stdout:Ign:3 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic Release.gpg 2021-03-11T00:39:55.651 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:8 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2021-03-11T00:39:55.671 INFO:teuthology.orchestra.run.smithi124.stdout:Hit:4 http://security.ubuntu.com/ubuntu bionic-security InRelease 2021-03-11T00:39:55.671 INFO:teuthology.orchestra.run.smithi124.stdout:Hit:5 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-03-11T00:39:55.672 INFO:teuthology.orchestra.run.smithi124.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 Packages [15.8 kB] 2021-03-11T00:39:55.696 INFO:teuthology.orchestra.run.smithi124.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,085 B] 2021-03-11T00:39:55.731 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2021-03-11T00:39:55.732 INFO:teuthology.orchestra.run.smithi124.stdout:Hit:8 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2021-03-11T00:39:55.810 INFO:teuthology.orchestra.run.smithi124.stdout:Hit:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2021-03-11T00:39:56.176 INFO:teuthology.orchestra.run.smithi119.stdout:Fetched 26.6 kB in 1s (37.9 kB/s) 2021-03-11T00:39:56.275 INFO:teuthology.orchestra.run.smithi124.stdout:Fetched 26.6 kB in 1s (37.8 kB/s) 2021-03-11T00:39:57.418 INFO:teuthology.orchestra.run.smithi119.stdout:Reading package lists... 2021-03-11T00:39:57.466 DEBUG:teuthology.orchestra.run.smithi119:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.9-108-g1ff7d244-1bionic cephadm=15.2.9-108-g1ff7d244-1bionic ceph-mds=15.2.9-108-g1ff7d244-1bionic ceph-mgr=15.2.9-108-g1ff7d244-1bionic ceph-common=15.2.9-108-g1ff7d244-1bionic ceph-fuse=15.2.9-108-g1ff7d244-1bionic ceph-test=15.2.9-108-g1ff7d244-1bionic radosgw=15.2.9-108-g1ff7d244-1bionic python3-rados=15.2.9-108-g1ff7d244-1bionic python3-rgw=15.2.9-108-g1ff7d244-1bionic python3-cephfs=15.2.9-108-g1ff7d244-1bionic python3-rbd=15.2.9-108-g1ff7d244-1bionic libcephfs2=15.2.9-108-g1ff7d244-1bionic libcephfs-dev=15.2.9-108-g1ff7d244-1bionic librados2=15.2.9-108-g1ff7d244-1bionic librbd1=15.2.9-108-g1ff7d244-1bionic rbd-fuse=15.2.9-108-g1ff7d244-1bionic 2021-03-11T00:39:57.534 INFO:teuthology.orchestra.run.smithi124.stdout:Reading package lists... 2021-03-11T00:39:57.546 INFO:teuthology.orchestra.run.smithi119.stdout:Reading package lists... 2021-03-11T00:39:57.588 DEBUG:teuthology.orchestra.run.smithi124:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.9-108-g1ff7d244-1bionic cephadm=15.2.9-108-g1ff7d244-1bionic ceph-mds=15.2.9-108-g1ff7d244-1bionic ceph-mgr=15.2.9-108-g1ff7d244-1bionic ceph-common=15.2.9-108-g1ff7d244-1bionic ceph-fuse=15.2.9-108-g1ff7d244-1bionic ceph-test=15.2.9-108-g1ff7d244-1bionic radosgw=15.2.9-108-g1ff7d244-1bionic python3-rados=15.2.9-108-g1ff7d244-1bionic python3-rgw=15.2.9-108-g1ff7d244-1bionic python3-cephfs=15.2.9-108-g1ff7d244-1bionic python3-rbd=15.2.9-108-g1ff7d244-1bionic libcephfs2=15.2.9-108-g1ff7d244-1bionic libcephfs-dev=15.2.9-108-g1ff7d244-1bionic librados2=15.2.9-108-g1ff7d244-1bionic librbd1=15.2.9-108-g1ff7d244-1bionic rbd-fuse=15.2.9-108-g1ff7d244-1bionic 2021-03-11T00:39:57.680 INFO:teuthology.orchestra.run.smithi124.stdout:Reading package lists... 2021-03-11T00:39:57.776 INFO:teuthology.orchestra.run.smithi119.stdout:Building dependency tree... 2021-03-11T00:39:57.777 INFO:teuthology.orchestra.run.smithi119.stdout:Reading state information... 2021-03-11T00:39:57.944 INFO:teuthology.orchestra.run.smithi119.stdout:The following additional packages will be installed: 2021-03-11T00:39:57.944 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-03-11T00:39:57.945 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-03-11T00:39:57.945 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook ceph-mon ceph-osd jq libjq1 libleveldb1v5 liblttng-ust0 2021-03-11T00:39:57.945 INFO:teuthology.orchestra.run.smithi119.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2021-03-11T00:39:57.945 INFO:teuthology.orchestra.run.smithi119.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2021-03-11T00:39:57.946 INFO:teuthology.orchestra.run.smithi119.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2021-03-11T00:39:57.946 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-03-11T00:39:57.947 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2021-03-11T00:39:57.947 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy3 python3-cryptography python3-cycler python3-dateutil 2021-03-11T00:39:57.947 INFO:teuthology.orchestra.run.smithi119.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2021-03-11T00:39:57.947 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2 python3-joblib python3-json-pointer python3-jsonpatch 2021-03-11T00:39:57.948 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2021-03-11T00:39:57.948 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2021-03-11T00:39:57.948 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2021-03-11T00:39:57.948 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2021-03-11T00:39:57.948 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2021-03-11T00:39:57.949 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pytest 2021-03-11T00:39:57.949 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2021-03-11T00:39:57.949 INFO:teuthology.orchestra.run.smithi119.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2021-03-11T00:39:57.949 INFO:teuthology.orchestra.run.smithi119.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2021-03-11T00:39:57.950 INFO:teuthology.orchestra.run.smithi119.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2021-03-11T00:39:57.952 INFO:teuthology.orchestra.run.smithi119.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2021-03-11T00:39:57.953 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2021-03-11T00:39:57.957 INFO:teuthology.orchestra.run.smithi119.stdout:Suggested packages: 2021-03-11T00:39:57.961 INFO:teuthology.orchestra.run.smithi119.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2021-03-11T00:39:57.961 INFO:teuthology.orchestra.run.smithi119.stdout: python-pytest-xdist python-mock python-cryptography-doc 2021-03-11T00:39:57.962 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2021-03-11T00:39:57.964 INFO:teuthology.orchestra.run.smithi119.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2021-03-11T00:39:57.966 INFO:teuthology.orchestra.run.smithi119.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2021-03-11T00:39:57.966 INFO:teuthology.orchestra.run.smithi119.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2021-03-11T00:39:57.970 INFO:teuthology.orchestra.run.smithi119.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2021-03-11T00:39:57.971 INFO:teuthology.orchestra.run.smithi119.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2021-03-11T00:39:57.971 INFO:teuthology.orchestra.run.smithi119.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2021-03-11T00:39:57.972 INFO:teuthology.orchestra.run.smithi119.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2021-03-11T00:39:57.975 INFO:teuthology.orchestra.run.smithi119.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2021-03-11T00:39:57.975 INFO:teuthology.orchestra.run.smithi119.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2021-03-11T00:39:57.976 INFO:teuthology.orchestra.run.smithi119.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2021-03-11T00:39:57.978 INFO:teuthology.orchestra.run.smithi119.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2021-03-11T00:39:57.979 INFO:teuthology.orchestra.run.smithi119.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2021-03-11T00:39:57.999 INFO:teuthology.orchestra.run.smithi124.stdout:Building dependency tree... 2021-03-11T00:39:58.000 INFO:teuthology.orchestra.run.smithi124.stdout:Reading state information... 2021-03-11T00:39:58.044 INFO:teuthology.orchestra.run.smithi119.stdout:The following NEW packages will be installed: 2021-03-11T00:39:58.044 INFO:teuthology.orchestra.run.smithi119.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2021-03-11T00:39:58.044 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-03-11T00:39:58.045 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-03-11T00:39:58.045 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm jq libcephfs-dev 2021-03-11T00:39:58.047 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2 libjq1 libleveldb1v5 liblttng-ust0 liboath0 libonig4 librabbitmq4 2021-03-11T00:39:58.047 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1 librdkafka1 librgw2 libsnappy1v5 libxslt1.1 nvme-cli 2021-03-11T00:39:58.048 INFO:teuthology.orchestra.run.smithi119.stdout: python-attr python-funcsigs python-pastedeploy-tpl python-pluggy python-py 2021-03-11T00:39:58.048 INFO:teuthology.orchestra.run.smithi119.stdout: python-pytest python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-03-11T00:39:58.048 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs 2021-03-11T00:39:58.048 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi-backend python3-cherrypy3 python3-cryptography python3-cycler 2021-03-11T00:39:58.048 INFO:teuthology.orchestra.run.smithi119.stdout: python3-dateutil python3-decorator python3-fasteners python3-html5lib 2021-03-11T00:39:58.049 INFO:teuthology.orchestra.run.smithi119.stdout: python3-httplib2 python3-jinja2 python3-joblib python3-json-pointer 2021-03-11T00:39:58.049 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch python3-jwt python3-kubernetes python3-logutils 2021-03-11T00:39:58.049 INFO:teuthology.orchestra.run.smithi119.stdout: python3-lxml python3-mako python3-markupsafe python3-matplotlib 2021-03-11T00:39:58.049 INFO:teuthology.orchestra.run.smithi119.stdout: python3-monotonic python3-numpy python3-oauth2client python3-olefile 2021-03-11T00:39:58.050 INFO:teuthology.orchestra.run.smithi119.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2021-03-11T00:39:58.050 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan python3-pil python3-pluggy python3-prettytable python3-py 2021-03-11T00:39:58.051 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyparsing 2021-03-11T00:39:58.051 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2021-03-11T00:39:58.051 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2021-03-11T00:39:58.052 INFO:teuthology.orchestra.run.smithi119.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2021-03-11T00:39:58.052 INFO:teuthology.orchestra.run.smithi119.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2021-03-11T00:39:58.053 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2021-03-11T00:39:58.053 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2021-03-11T00:39:58.053 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2021-03-11T00:39:58.053 INFO:teuthology.orchestra.run.smithi119.stdout:The following packages will be upgraded: 2021-03-11T00:39:58.054 INFO:teuthology.orchestra.run.smithi119.stdout: librados2 librbd1 2021-03-11T00:39:58.192 INFO:teuthology.orchestra.run.smithi119.stdout:2 upgraded, 111 newly installed, 0 to remove and 16 not upgraded. 2021-03-11T00:39:58.192 INFO:teuthology.orchestra.run.smithi119.stdout:Need to get 156 MB of archives. 2021-03-11T00:39:58.192 INFO:teuthology.orchestra.run.smithi119.stdout:After this operation, 725 MB of additional disk space will be used. 2021-03-11T00:39:58.193 INFO:teuthology.orchestra.run.smithi119.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 15.2.9-108-g1ff7d244-1bionic [1,641 kB] 2021-03-11T00:39:58.215 INFO:teuthology.orchestra.run.smithi124.stdout:The following additional packages will be installed: 2021-03-11T00:39:58.216 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-03-11T00:39:58.216 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-03-11T00:39:58.217 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-rook ceph-mon ceph-osd jq libjq1 libleveldb1v5 liblttng-ust0 2021-03-11T00:39:58.217 INFO:teuthology.orchestra.run.smithi124.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2021-03-11T00:39:58.217 INFO:teuthology.orchestra.run.smithi124.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2021-03-11T00:39:58.218 INFO:teuthology.orchestra.run.smithi124.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2021-03-11T00:39:58.218 INFO:teuthology.orchestra.run.smithi124.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-03-11T00:39:58.218 INFO:teuthology.orchestra.run.smithi124.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2021-03-11T00:39:58.218 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cherrypy3 python3-cryptography python3-cycler python3-dateutil 2021-03-11T00:39:58.219 INFO:teuthology.orchestra.run.smithi124.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2021-03-11T00:39:58.219 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jinja2 python3-joblib python3-json-pointer python3-jsonpatch 2021-03-11T00:39:58.219 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2021-03-11T00:39:58.219 INFO:teuthology.orchestra.run.smithi124.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2021-03-11T00:39:58.220 INFO:teuthology.orchestra.run.smithi124.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2021-03-11T00:39:58.220 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2021-03-11T00:39:58.220 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2021-03-11T00:39:58.221 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pytest 2021-03-11T00:39:58.221 INFO:teuthology.orchestra.run.smithi124.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2021-03-11T00:39:58.221 INFO:teuthology.orchestra.run.smithi124.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2021-03-11T00:39:58.221 INFO:teuthology.orchestra.run.smithi124.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2021-03-11T00:39:58.221 INFO:teuthology.orchestra.run.smithi124.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2021-03-11T00:39:58.222 INFO:teuthology.orchestra.run.smithi124.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2021-03-11T00:39:58.222 INFO:teuthology.orchestra.run.smithi124.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2021-03-11T00:39:58.223 INFO:teuthology.orchestra.run.smithi124.stdout:Suggested packages: 2021-03-11T00:39:58.223 INFO:teuthology.orchestra.run.smithi124.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2021-03-11T00:39:58.223 INFO:teuthology.orchestra.run.smithi124.stdout: python-pytest-xdist python-mock python-cryptography-doc 2021-03-11T00:39:58.223 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2021-03-11T00:39:58.224 INFO:teuthology.orchestra.run.smithi124.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2021-03-11T00:39:58.224 INFO:teuthology.orchestra.run.smithi124.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2021-03-11T00:39:58.224 INFO:teuthology.orchestra.run.smithi124.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2021-03-11T00:39:58.224 INFO:teuthology.orchestra.run.smithi124.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2021-03-11T00:39:58.225 INFO:teuthology.orchestra.run.smithi124.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2021-03-11T00:39:58.240 INFO:teuthology.orchestra.run.smithi124.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2021-03-11T00:39:58.240 INFO:teuthology.orchestra.run.smithi124.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2021-03-11T00:39:58.240 INFO:teuthology.orchestra.run.smithi124.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2021-03-11T00:39:58.240 INFO:teuthology.orchestra.run.smithi124.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2021-03-11T00:39:58.241 INFO:teuthology.orchestra.run.smithi124.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2021-03-11T00:39:58.241 INFO:teuthology.orchestra.run.smithi124.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2021-03-11T00:39:58.241 INFO:teuthology.orchestra.run.smithi124.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2021-03-11T00:39:58.242 INFO:teuthology.orchestra.run.smithi119.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2021-03-11T00:39:58.324 INFO:teuthology.orchestra.run.smithi119.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2021-03-11T00:39:58.360 INFO:teuthology.orchestra.run.smithi124.stdout:The following NEW packages will be installed: 2021-03-11T00:39:58.361 INFO:teuthology.orchestra.run.smithi124.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2021-03-11T00:39:58.361 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-03-11T00:39:58.361 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-03-11T00:39:58.362 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm jq libcephfs-dev 2021-03-11T00:39:58.362 INFO:teuthology.orchestra.run.smithi124.stdout: libcephfs2 libjq1 libleveldb1v5 liblttng-ust0 liboath0 libonig4 librabbitmq4 2021-03-11T00:39:58.363 INFO:teuthology.orchestra.run.smithi124.stdout: libradosstriper1 librdkafka1 librgw2 libsnappy1v5 libxslt1.1 nvme-cli 2021-03-11T00:39:58.364 INFO:teuthology.orchestra.run.smithi124.stdout: python-attr python-funcsigs python-pastedeploy-tpl python-pluggy python-py 2021-03-11T00:39:58.364 INFO:teuthology.orchestra.run.smithi124.stdout: python-pytest python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-03-11T00:39:58.364 INFO:teuthology.orchestra.run.smithi124.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs 2021-03-11T00:39:58.364 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cffi-backend python3-cherrypy3 python3-cryptography python3-cycler 2021-03-11T00:39:58.365 INFO:teuthology.orchestra.run.smithi124.stdout: python3-dateutil python3-decorator python3-fasteners python3-html5lib 2021-03-11T00:39:58.365 INFO:teuthology.orchestra.run.smithi124.stdout: python3-httplib2 python3-jinja2 python3-joblib python3-json-pointer 2021-03-11T00:39:58.365 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jsonpatch python3-jwt python3-kubernetes python3-logutils 2021-03-11T00:39:58.365 INFO:teuthology.orchestra.run.smithi124.stdout: python3-lxml python3-mako python3-markupsafe python3-matplotlib 2021-03-11T00:39:58.365 INFO:teuthology.orchestra.run.smithi124.stdout: python3-monotonic python3-numpy python3-oauth2client python3-olefile 2021-03-11T00:39:58.366 INFO:teuthology.orchestra.run.smithi124.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2021-03-11T00:39:58.366 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pecan python3-pil python3-pluggy python3-prettytable python3-py 2021-03-11T00:39:58.366 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyparsing 2021-03-11T00:39:58.367 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2021-03-11T00:39:58.367 INFO:teuthology.orchestra.run.smithi124.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2021-03-11T00:39:58.367 INFO:teuthology.orchestra.run.smithi124.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2021-03-11T00:39:58.367 INFO:teuthology.orchestra.run.smithi124.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2021-03-11T00:39:58.368 INFO:teuthology.orchestra.run.smithi124.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2021-03-11T00:39:58.368 INFO:teuthology.orchestra.run.smithi124.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2021-03-11T00:39:58.368 INFO:teuthology.orchestra.run.smithi124.stdout: python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2021-03-11T00:39:58.369 INFO:teuthology.orchestra.run.smithi124.stdout:The following packages will be upgraded: 2021-03-11T00:39:58.369 INFO:teuthology.orchestra.run.smithi124.stdout: librados2 librbd1 2021-03-11T00:39:58.416 INFO:teuthology.orchestra.run.smithi119.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 15.2.9-108-g1ff7d244-1bionic [3,060 kB] 2021-03-11T00:39:58.516 INFO:teuthology.orchestra.run.smithi124.stdout:2 upgraded, 111 newly installed, 0 to remove and 16 not upgraded. 2021-03-11T00:39:58.516 INFO:teuthology.orchestra.run.smithi124.stdout:Need to get 156 MB of archives. 2021-03-11T00:39:58.517 INFO:teuthology.orchestra.run.smithi124.stdout:After this operation, 725 MB of additional disk space will be used. 2021-03-11T00:39:58.517 INFO:teuthology.orchestra.run.smithi124.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 15.2.9-108-g1ff7d244-1bionic [1,641 kB] 2021-03-11T00:39:58.557 INFO:teuthology.orchestra.run.smithi124.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2021-03-11T00:39:58.577 INFO:teuthology.orchestra.run.smithi119.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2021-03-11T00:39:58.592 INFO:teuthology.orchestra.run.smithi119.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2021-03-11T00:39:58.598 INFO:teuthology.orchestra.run.smithi119.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 15.2.9-108-g1ff7d244-1bionic [436 kB] 2021-03-11T00:39:58.608 INFO:teuthology.orchestra.run.smithi119.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2021-03-11T00:39:58.619 INFO:teuthology.orchestra.run.smithi119.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 15.2.9-108-g1ff7d244-1bionic [345 kB] 2021-03-11T00:39:58.628 INFO:teuthology.orchestra.run.smithi119.stdout:Get:10 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 15.2.9-108-g1ff7d244-1bionic [20.7 kB] 2021-03-11T00:39:58.629 INFO:teuthology.orchestra.run.smithi119.stdout:Get:11 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 15.2.9-108-g1ff7d244-1bionic [133 kB] 2021-03-11T00:39:58.641 INFO:teuthology.orchestra.run.smithi119.stdout:Get:12 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 15.2.9-108-g1ff7d244-1bionic [29.0 kB] 2021-03-11T00:39:58.642 INFO:teuthology.orchestra.run.smithi119.stdout:Get:13 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 15.2.9-108-g1ff7d244-1bionic [326 kB] 2021-03-11T00:39:58.647 INFO:teuthology.orchestra.run.smithi124.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2021-03-11T00:39:58.652 INFO:teuthology.orchestra.run.smithi119.stdout:Get:14 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 15.2.9-108-g1ff7d244-1bionic [2,902 kB] 2021-03-11T00:39:58.726 INFO:teuthology.orchestra.run.smithi119.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2021-03-11T00:39:58.727 INFO:teuthology.orchestra.run.smithi124.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 15.2.9-108-g1ff7d244-1bionic [3,060 kB] 2021-03-11T00:39:58.730 INFO:teuthology.orchestra.run.smithi119.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2021-03-11T00:39:58.775 INFO:teuthology.orchestra.run.smithi119.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2021-03-11T00:39:58.783 INFO:teuthology.orchestra.run.smithi119.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2021-03-11T00:39:58.793 INFO:teuthology.orchestra.run.smithi119.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2021-03-11T00:39:58.797 INFO:teuthology.orchestra.run.smithi119.stdout:Get:20 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 15.2.9-108-g1ff7d244-1bionic [99.6 kB] 2021-03-11T00:39:58.800 INFO:teuthology.orchestra.run.smithi119.stdout:Get:21 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 15.2.9-108-g1ff7d244-1bionic [338 kB] 2021-03-11T00:39:58.804 INFO:teuthology.orchestra.run.smithi119.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2021-03-11T00:39:58.811 INFO:teuthology.orchestra.run.smithi119.stdout:Get:23 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 15.2.9-108-g1ff7d244-1bionic [18.8 MB] 2021-03-11T00:39:58.812 INFO:teuthology.orchestra.run.smithi119.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2021-03-11T00:39:58.848 INFO:teuthology.orchestra.run.smithi119.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2021-03-11T00:39:58.850 INFO:teuthology.orchestra.run.smithi119.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2021-03-11T00:39:58.852 INFO:teuthology.orchestra.run.smithi119.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2021-03-11T00:39:58.865 INFO:teuthology.orchestra.run.smithi119.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2021-03-11T00:39:58.900 INFO:teuthology.orchestra.run.smithi124.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2021-03-11T00:39:58.908 INFO:teuthology.orchestra.run.smithi119.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2021-03-11T00:39:58.909 INFO:teuthology.orchestra.run.smithi119.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2021-03-11T00:39:58.909 INFO:teuthology.orchestra.run.smithi119.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2021-03-11T00:39:58.911 INFO:teuthology.orchestra.run.smithi124.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2021-03-11T00:39:58.913 INFO:teuthology.orchestra.run.smithi119.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2021-03-11T00:39:58.914 INFO:teuthology.orchestra.run.smithi119.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2021-03-11T00:39:58.922 INFO:teuthology.orchestra.run.smithi124.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2021-03-11T00:39:58.988 INFO:teuthology.orchestra.run.smithi119.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2021-03-11T00:39:58.990 INFO:teuthology.orchestra.run.smithi119.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2021-03-11T00:39:58.993 INFO:teuthology.orchestra.run.smithi119.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2021-03-11T00:39:58.995 INFO:teuthology.orchestra.run.smithi119.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2021-03-11T00:39:58.996 INFO:teuthology.orchestra.run.smithi119.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2021-03-11T00:39:59.030 INFO:teuthology.orchestra.run.smithi124.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 15.2.9-108-g1ff7d244-1bionic [436 kB] 2021-03-11T00:39:59.067 INFO:teuthology.orchestra.run.smithi119.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2021-03-11T00:39:59.068 INFO:teuthology.orchestra.run.smithi119.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2021-03-11T00:39:59.068 INFO:teuthology.orchestra.run.smithi119.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2021-03-11T00:39:59.069 INFO:teuthology.orchestra.run.smithi124.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2021-03-11T00:39:59.069 INFO:teuthology.orchestra.run.smithi119.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2021-03-11T00:39:59.071 INFO:teuthology.orchestra.run.smithi124.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2021-03-11T00:39:59.072 INFO:teuthology.orchestra.run.smithi124.stdout:Get:11 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 15.2.9-108-g1ff7d244-1bionic [345 kB] 2021-03-11T00:39:59.073 INFO:teuthology.orchestra.run.smithi119.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1ubuntu0.1 [174 kB] 2021-03-11T00:39:59.085 INFO:teuthology.orchestra.run.smithi124.stdout:Get:12 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 15.2.9-108-g1ff7d244-1bionic [20.7 kB] 2021-03-11T00:39:59.086 INFO:teuthology.orchestra.run.smithi124.stdout:Get:13 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 15.2.9-108-g1ff7d244-1bionic [133 kB] 2021-03-11T00:39:59.090 INFO:teuthology.orchestra.run.smithi124.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2021-03-11T00:39:59.097 INFO:teuthology.orchestra.run.smithi124.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2021-03-11T00:39:59.106 INFO:teuthology.orchestra.run.smithi124.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2021-03-11T00:39:59.119 INFO:teuthology.orchestra.run.smithi124.stdout:Get:17 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 15.2.9-108-g1ff7d244-1bionic [29.0 kB] 2021-03-11T00:39:59.120 INFO:teuthology.orchestra.run.smithi124.stdout:Get:18 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 15.2.9-108-g1ff7d244-1bionic [326 kB] 2021-03-11T00:39:59.125 INFO:teuthology.orchestra.run.smithi124.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2021-03-11T00:39:59.130 INFO:teuthology.orchestra.run.smithi124.stdout:Get:20 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 15.2.9-108-g1ff7d244-1bionic [2,902 kB] 2021-03-11T00:39:59.137 INFO:teuthology.orchestra.run.smithi124.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2021-03-11T00:39:59.147 INFO:teuthology.orchestra.run.smithi119.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2021-03-11T00:39:59.151 INFO:teuthology.orchestra.run.smithi119.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2021-03-11T00:39:59.151 INFO:teuthology.orchestra.run.smithi119.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2021-03-11T00:39:59.152 INFO:teuthology.orchestra.run.smithi119.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2021-03-11T00:39:59.169 INFO:teuthology.orchestra.run.smithi124.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2021-03-11T00:39:59.173 INFO:teuthology.orchestra.run.smithi124.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2021-03-11T00:39:59.176 INFO:teuthology.orchestra.run.smithi124.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2021-03-11T00:39:59.228 INFO:teuthology.orchestra.run.smithi124.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2021-03-11T00:39:59.229 INFO:teuthology.orchestra.run.smithi124.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2021-03-11T00:39:59.230 INFO:teuthology.orchestra.run.smithi124.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2021-03-11T00:39:59.230 INFO:teuthology.orchestra.run.smithi124.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2021-03-11T00:39:59.234 INFO:teuthology.orchestra.run.smithi124.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2021-03-11T00:39:59.235 INFO:teuthology.orchestra.run.smithi124.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2021-03-11T00:39:59.237 INFO:teuthology.orchestra.run.smithi119.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2021-03-11T00:39:59.238 INFO:teuthology.orchestra.run.smithi119.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2021-03-11T00:39:59.306 INFO:teuthology.orchestra.run.smithi124.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2021-03-11T00:39:59.309 INFO:teuthology.orchestra.run.smithi124.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2021-03-11T00:39:59.312 INFO:teuthology.orchestra.run.smithi124.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2021-03-11T00:39:59.314 INFO:teuthology.orchestra.run.smithi124.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2021-03-11T00:39:59.384 INFO:teuthology.orchestra.run.smithi124.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2021-03-11T00:39:59.399 INFO:teuthology.orchestra.run.smithi124.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2021-03-11T00:39:59.400 INFO:teuthology.orchestra.run.smithi124.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2021-03-11T00:39:59.400 INFO:teuthology.orchestra.run.smithi124.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2021-03-11T00:39:59.400 INFO:teuthology.orchestra.run.smithi124.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2021-03-11T00:39:59.403 INFO:teuthology.orchestra.run.smithi124.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1ubuntu0.1 [174 kB] 2021-03-11T00:39:59.441 INFO:teuthology.orchestra.run.smithi124.stdout:Get:41 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 15.2.9-108-g1ff7d244-1bionic [99.6 kB] 2021-03-11T00:39:59.444 INFO:teuthology.orchestra.run.smithi124.stdout:Get:42 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 15.2.9-108-g1ff7d244-1bionic [338 kB] 2021-03-11T00:39:59.455 INFO:teuthology.orchestra.run.smithi124.stdout:Get:43 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 15.2.9-108-g1ff7d244-1bionic [18.8 MB] 2021-03-11T00:39:59.462 INFO:teuthology.orchestra.run.smithi124.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2021-03-11T00:39:59.467 INFO:teuthology.orchestra.run.smithi124.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2021-03-11T00:39:59.467 INFO:teuthology.orchestra.run.smithi124.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2021-03-11T00:39:59.469 INFO:teuthology.orchestra.run.smithi124.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2021-03-11T00:39:59.527 INFO:teuthology.orchestra.run.smithi119.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2021-03-11T00:39:59.546 INFO:teuthology.orchestra.run.smithi124.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2021-03-11T00:39:59.547 INFO:teuthology.orchestra.run.smithi124.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2021-03-11T00:39:59.586 INFO:teuthology.orchestra.run.smithi119.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2021-03-11T00:39:59.587 INFO:teuthology.orchestra.run.smithi119.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2021-03-11T00:39:59.610 INFO:teuthology.orchestra.run.smithi119.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2021-03-11T00:39:59.611 INFO:teuthology.orchestra.run.smithi119.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2021-03-11T00:39:59.611 INFO:teuthology.orchestra.run.smithi119.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.3 [28.8 kB] 2021-03-11T00:39:59.612 INFO:teuthology.orchestra.run.smithi119.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2021-03-11T00:39:59.612 INFO:teuthology.orchestra.run.smithi119.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2021-03-11T00:39:59.613 INFO:teuthology.orchestra.run.smithi119.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2021-03-11T00:39:59.614 INFO:teuthology.orchestra.run.smithi119.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2021-03-11T00:39:59.687 INFO:teuthology.orchestra.run.smithi119.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2021-03-11T00:39:59.690 INFO:teuthology.orchestra.run.smithi119.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2021-03-11T00:39:59.750 INFO:teuthology.orchestra.run.smithi119.stdout:Get:62 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 15.2.9-108-g1ff7d244-1bionic [5,173 kB] 2021-03-11T00:39:59.767 INFO:teuthology.orchestra.run.smithi119.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2021-03-11T00:39:59.768 INFO:teuthology.orchestra.run.smithi119.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2021-03-11T00:39:59.772 INFO:teuthology.orchestra.run.smithi119.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2021-03-11T00:39:59.773 INFO:teuthology.orchestra.run.smithi119.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2021-03-11T00:39:59.773 INFO:teuthology.orchestra.run.smithi119.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2021-03-11T00:39:59.775 INFO:teuthology.orchestra.run.smithi119.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2021-03-11T00:39:59.777 INFO:teuthology.orchestra.run.smithi119.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2021-03-11T00:39:59.777 INFO:teuthology.orchestra.run.smithi119.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2021-03-11T00:39:59.827 INFO:teuthology.orchestra.run.smithi124.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2021-03-11T00:39:59.846 INFO:teuthology.orchestra.run.smithi119.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2021-03-11T00:39:59.853 INFO:teuthology.orchestra.run.smithi119.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2021-03-11T00:39:59.876 INFO:teuthology.orchestra.run.smithi124.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2021-03-11T00:39:59.877 INFO:teuthology.orchestra.run.smithi124.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2021-03-11T00:39:59.903 INFO:teuthology.orchestra.run.smithi124.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2021-03-11T00:39:59.903 INFO:teuthology.orchestra.run.smithi124.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2021-03-11T00:39:59.904 INFO:teuthology.orchestra.run.smithi124.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.3 [28.8 kB] 2021-03-11T00:39:59.904 INFO:teuthology.orchestra.run.smithi124.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2021-03-11T00:39:59.905 INFO:teuthology.orchestra.run.smithi124.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2021-03-11T00:39:59.906 INFO:teuthology.orchestra.run.smithi124.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2021-03-11T00:39:59.907 INFO:teuthology.orchestra.run.smithi124.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2021-03-11T00:39:59.926 INFO:teuthology.orchestra.run.smithi119.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.1 [184 kB] 2021-03-11T00:39:59.933 INFO:teuthology.orchestra.run.smithi119.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2021-03-11T00:39:59.934 INFO:teuthology.orchestra.run.smithi119.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2021-03-11T00:39:59.935 INFO:teuthology.orchestra.run.smithi119.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2021-03-11T00:39:59.935 INFO:teuthology.orchestra.run.smithi119.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2021-03-11T00:39:59.938 INFO:teuthology.orchestra.run.smithi119.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2021-03-11T00:39:59.940 INFO:teuthology.orchestra.run.smithi119.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2021-03-11T00:39:59.941 INFO:teuthology.orchestra.run.smithi119.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2021-03-11T00:39:59.985 INFO:teuthology.orchestra.run.smithi124.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2021-03-11T00:39:59.985 INFO:teuthology.orchestra.run.smithi124.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2021-03-11T00:39:59.988 INFO:teuthology.orchestra.run.smithi124.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2021-03-11T00:39:59.988 INFO:teuthology.orchestra.run.smithi124.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2021-03-11T00:39:59.990 INFO:teuthology.orchestra.run.smithi124.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2021-03-11T00:39:59.990 INFO:teuthology.orchestra.run.smithi124.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2021-03-11T00:39:59.991 INFO:teuthology.orchestra.run.smithi124.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2021-03-11T00:40:00.007 INFO:teuthology.orchestra.run.smithi119.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2021-03-11T00:40:00.007 INFO:teuthology.orchestra.run.smithi119.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2021-03-11T00:40:00.020 INFO:teuthology.orchestra.run.smithi119.stdout:Get:83 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 15.2.9-108-g1ff7d244-1bionic [161 kB] 2021-03-11T00:40:00.022 INFO:teuthology.orchestra.run.smithi119.stdout:Get:84 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 15.2.9-108-g1ff7d244-1bionic [1,284 kB] 2021-03-11T00:40:00.063 INFO:teuthology.orchestra.run.smithi124.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2021-03-11T00:40:00.064 INFO:teuthology.orchestra.run.smithi124.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2021-03-11T00:40:00.064 INFO:teuthology.orchestra.run.smithi124.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2021-03-11T00:40:00.073 INFO:teuthology.orchestra.run.smithi119.stdout:Get:85 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 15.2.9-108-g1ff7d244-1bionic [5,944 kB] 2021-03-11T00:40:00.085 INFO:teuthology.orchestra.run.smithi119.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.3 [1,097 kB] 2021-03-11T00:40:00.102 INFO:teuthology.orchestra.run.smithi119.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2021-03-11T00:40:00.103 INFO:teuthology.orchestra.run.smithi119.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2021-03-11T00:40:00.103 INFO:teuthology.orchestra.run.smithi119.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2021-03-11T00:40:00.141 INFO:teuthology.orchestra.run.smithi124.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2021-03-11T00:40:00.147 INFO:teuthology.orchestra.run.smithi124.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2021-03-11T00:40:00.154 INFO:teuthology.orchestra.run.smithi124.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.1 [184 kB] 2021-03-11T00:40:00.156 INFO:teuthology.orchestra.run.smithi124.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2021-03-11T00:40:00.157 INFO:teuthology.orchestra.run.smithi124.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2021-03-11T00:40:00.157 INFO:teuthology.orchestra.run.smithi124.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2021-03-11T00:40:00.158 INFO:teuthology.orchestra.run.smithi124.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2021-03-11T00:40:00.195 INFO:teuthology.orchestra.run.smithi119.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2021-03-11T00:40:00.195 INFO:teuthology.orchestra.run.smithi119.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2021-03-11T00:40:00.196 INFO:teuthology.orchestra.run.smithi119.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.4 [329 kB] 2021-03-11T00:40:00.200 INFO:teuthology.orchestra.run.smithi119.stdout:Get:93 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2021-03-11T00:40:00.200 INFO:teuthology.orchestra.run.smithi119.stdout:Get:94 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2021-03-11T00:40:00.220 INFO:teuthology.orchestra.run.smithi124.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2021-03-11T00:40:00.222 INFO:teuthology.orchestra.run.smithi124.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2021-03-11T00:40:00.223 INFO:teuthology.orchestra.run.smithi124.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2021-03-11T00:40:00.245 INFO:teuthology.orchestra.run.smithi119.stdout:Get:95 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2021-03-11T00:40:00.246 INFO:teuthology.orchestra.run.smithi119.stdout:Get:96 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2021-03-11T00:40:00.263 INFO:teuthology.orchestra.run.smithi119.stdout:Get:97 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2021-03-11T00:40:00.263 INFO:teuthology.orchestra.run.smithi119.stdout:Get:98 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2021-03-11T00:40:00.298 INFO:teuthology.orchestra.run.smithi124.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2021-03-11T00:40:00.299 INFO:teuthology.orchestra.run.smithi124.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2021-03-11T00:40:00.300 INFO:teuthology.orchestra.run.smithi124.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.3 [1,097 kB] 2021-03-11T00:40:00.317 INFO:teuthology.orchestra.run.smithi124.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2021-03-11T00:40:00.317 INFO:teuthology.orchestra.run.smithi124.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2021-03-11T00:40:00.318 INFO:teuthology.orchestra.run.smithi124.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2021-03-11T00:40:00.359 INFO:teuthology.orchestra.run.smithi119.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 15.2.9-108-g1ff7d244-1bionic [22.8 MB] 2021-03-11T00:40:00.412 INFO:teuthology.orchestra.run.smithi124.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2021-03-11T00:40:00.412 INFO:teuthology.orchestra.run.smithi124.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2021-03-11T00:40:00.413 INFO:teuthology.orchestra.run.smithi124.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.4 [329 kB] 2021-03-11T00:40:00.459 INFO:teuthology.orchestra.run.smithi124.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2021-03-11T00:40:00.460 INFO:teuthology.orchestra.run.smithi124.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2021-03-11T00:40:00.460 INFO:teuthology.orchestra.run.smithi124.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2021-03-11T00:40:00.461 INFO:teuthology.orchestra.run.smithi124.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2021-03-11T00:40:00.463 INFO:teuthology.orchestra.run.smithi124.stdout:Get:93 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2021-03-11T00:40:00.466 INFO:teuthology.orchestra.run.smithi124.stdout:Get:94 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2021-03-11T00:40:01.329 INFO:teuthology.orchestra.run.smithi124.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 15.2.9-108-g1ff7d244-1bionic [5,173 kB] 2021-03-11T00:40:01.492 INFO:teuthology.orchestra.run.smithi119.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 15.2.9-108-g1ff7d244-1bionic [3,932 B] 2021-03-11T00:40:01.492 INFO:teuthology.orchestra.run.smithi119.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 15.2.9-108-g1ff7d244-1bionic [517 kB] 2021-03-11T00:40:01.505 INFO:teuthology.orchestra.run.smithi119.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 15.2.9-108-g1ff7d244-1bionic [1,854 kB] 2021-03-11T00:40:01.610 INFO:teuthology.orchestra.run.smithi119.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 15.2.9-108-g1ff7d244-1bionic [52.2 kB] 2021-03-11T00:40:01.612 INFO:teuthology.orchestra.run.smithi119.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 15.2.9-108-g1ff7d244-1bionic [57.9 kB] 2021-03-11T00:40:01.613 INFO:teuthology.orchestra.run.smithi119.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 15.2.9-108-g1ff7d244-1bionic [3,197 kB] 2021-03-11T00:40:01.763 INFO:teuthology.orchestra.run.smithi119.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-cloud all 15.2.9-108-g1ff7d244-1bionic [34.3 kB] 2021-03-11T00:40:01.764 INFO:teuthology.orchestra.run.smithi119.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 15.2.9-108-g1ff7d244-1bionic [7,893 kB] 2021-03-11T00:40:01.796 INFO:teuthology.orchestra.run.smithi124.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 15.2.9-108-g1ff7d244-1bionic [161 kB] 2021-03-11T00:40:01.842 INFO:teuthology.orchestra.run.smithi124.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 15.2.9-108-g1ff7d244-1bionic [1,284 kB] 2021-03-11T00:40:01.973 INFO:teuthology.orchestra.run.smithi124.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 15.2.9-108-g1ff7d244-1bionic [5,944 kB] 2021-03-11T00:40:02.164 INFO:teuthology.orchestra.run.smithi119.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 15.2.9-108-g1ff7d244-1bionic [4,048 B] 2021-03-11T00:40:02.165 INFO:teuthology.orchestra.run.smithi119.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 15.2.9-108-g1ff7d244-1bionic [22.5 kB] 2021-03-11T00:40:02.166 INFO:teuthology.orchestra.run.smithi119.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 15.2.9-108-g1ff7d244-1bionic [44.1 MB] 2021-03-11T00:40:02.536 INFO:teuthology.orchestra.run.smithi124.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 15.2.9-108-g1ff7d244-1bionic [22.8 MB] 2021-03-11T00:40:04.379 INFO:teuthology.orchestra.run.smithi119.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 15.2.9-108-g1ff7d244-1bionic [18.0 kB] 2021-03-11T00:40:04.380 INFO:teuthology.orchestra.run.smithi119.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 15.2.9-108-g1ff7d244-1bionic [8,880 kB] 2021-03-11T00:40:04.763 INFO:teuthology.orchestra.run.smithi124.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 15.2.9-108-g1ff7d244-1bionic [3,932 B] 2021-03-11T00:40:04.763 INFO:teuthology.orchestra.run.smithi124.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 15.2.9-108-g1ff7d244-1bionic [517 kB] 2021-03-11T00:40:04.766 INFO:teuthology.orchestra.run.smithi124.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 15.2.9-108-g1ff7d244-1bionic [1,854 kB] 2021-03-11T00:40:04.795 INFO:teuthology.orchestra.run.smithi119.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 15.2.9-108-g1ff7d244-1bionic [63.1 kB] 2021-03-11T00:40:04.853 INFO:teuthology.orchestra.run.smithi124.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 15.2.9-108-g1ff7d244-1bionic [52.2 kB] 2021-03-11T00:40:04.856 INFO:teuthology.orchestra.run.smithi124.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 15.2.9-108-g1ff7d244-1bionic [57.9 kB] 2021-03-11T00:40:04.858 INFO:teuthology.orchestra.run.smithi124.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 15.2.9-108-g1ff7d244-1bionic [3,197 kB] 2021-03-11T00:40:04.970 INFO:teuthology.orchestra.run.smithi124.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-cloud all 15.2.9-108-g1ff7d244-1bionic [34.3 kB] 2021-03-11T00:40:04.971 INFO:teuthology.orchestra.run.smithi124.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 15.2.9-108-g1ff7d244-1bionic [7,893 kB] 2021-03-11T00:40:05.246 INFO:teuthology.orchestra.run.smithi124.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 15.2.9-108-g1ff7d244-1bionic [4,048 B] 2021-03-11T00:40:05.247 INFO:teuthology.orchestra.run.smithi124.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 15.2.9-108-g1ff7d244-1bionic [22.5 kB] 2021-03-11T00:40:05.248 INFO:teuthology.orchestra.run.smithi124.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 15.2.9-108-g1ff7d244-1bionic [44.1 MB] 2021-03-11T00:40:05.489 INFO:teuthology.orchestra.run.smithi119.stdout:Fetched 156 MB in 7s (23.2 MB/s) 2021-03-11T00:40:05.628 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-uritemplate. 2021-03-11T00:40:05.667 INFO:teuthology.orchestra.run.smithi119.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102065 files and directories currently installed.) 2021-03-11T00:40:05.668 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../000-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2021-03-11T00:40:05.709 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2021-03-11T00:40:06.031 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2021-03-11T00:40:06.040 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../001-liblttng-ust0_2.10.1-1_amd64.deb ... 2021-03-11T00:40:06.146 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2021-03-11T00:40:06.486 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../002-librbd1_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:06.608 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking librbd1 (15.2.9-108-g1ff7d244-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-03-11T00:40:06.694 INFO:teuthology.orchestra.run.smithi124.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 15.2.9-108-g1ff7d244-1bionic [18.0 kB] 2021-03-11T00:40:06.696 INFO:teuthology.orchestra.run.smithi124.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 15.2.9-108-g1ff7d244-1bionic [8,880 kB] 2021-03-11T00:40:06.982 INFO:teuthology.orchestra.run.smithi124.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 15.2.9-108-g1ff7d244-1bionic [63.1 kB] 2021-03-11T00:40:07.090 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../003-librados2_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:07.216 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking librados2 (15.2.9-108-g1ff7d244-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-03-11T00:40:07.676 INFO:teuthology.orchestra.run.smithi124.stdout:Fetched 156 MB in 9s (18.2 MB/s) 2021-03-11T00:40:07.806 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-uritemplate. 2021-03-11T00:40:07.843 INFO:teuthology.orchestra.run.smithi124.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102065 files and directories currently installed.) 2021-03-11T00:40:07.846 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../000-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2021-03-11T00:40:07.866 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libcephfs2. 2021-03-11T00:40:07.876 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../004-libcephfs2_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:07.888 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2021-03-11T00:40:07.915 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libcephfs2 (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:08.201 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-rados. 2021-03-11T00:40:08.216 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2021-03-11T00:40:08.217 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../005-python3-rados_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:08.227 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../001-liblttng-ust0_2.10.1-1_amd64.deb ... 2021-03-11T00:40:08.251 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-rados (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:08.308 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2021-03-11T00:40:08.553 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-ceph-argparse. 2021-03-11T00:40:08.568 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../006-python3-ceph-argparse_15.2.9-108-g1ff7d244-1bionic_all.deb ... 2021-03-11T00:40:08.603 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-ceph-argparse (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:08.646 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../002-librbd1_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:08.753 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking librbd1 (15.2.9-108-g1ff7d244-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-03-11T00:40:08.914 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-cephfs. 2021-03-11T00:40:08.926 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../007-python3-cephfs_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:09.152 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-cephfs (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:09.275 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../003-librados2_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:09.407 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking librados2 (15.2.9-108-g1ff7d244-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-03-11T00:40:09.467 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-ceph-common. 2021-03-11T00:40:09.479 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../008-python3-ceph-common_15.2.9-108-g1ff7d244-1bionic_all.deb ... 2021-03-11T00:40:09.517 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-ceph-common (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:09.811 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-prettytable. 2021-03-11T00:40:09.823 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../009-python3-prettytable_0.7.2-3_all.deb ... 2021-03-11T00:40:09.871 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2021-03-11T00:40:10.060 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package libcephfs2. 2021-03-11T00:40:10.071 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../004-libcephfs2_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:10.089 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-rbd. 2021-03-11T00:40:10.100 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../010-python3-rbd_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:10.110 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking libcephfs2 (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:10.130 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-rbd (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:10.370 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-rados. 2021-03-11T00:40:10.383 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../005-python3-rados_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:10.434 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-rados (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:10.500 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package librabbitmq4:amd64. 2021-03-11T00:40:10.512 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../011-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2021-03-11T00:40:10.550 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-03-11T00:40:10.772 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-ceph-argparse. 2021-03-11T00:40:10.783 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../006-python3-ceph-argparse_15.2.9-108-g1ff7d244-1bionic_all.deb ... 2021-03-11T00:40:10.823 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-ceph-argparse (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:10.910 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package librdkafka1:amd64. 2021-03-11T00:40:10.922 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../012-librdkafka1_0.11.3-1build1_amd64.deb ... 2021-03-11T00:40:10.960 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2021-03-11T00:40:11.150 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-cephfs. 2021-03-11T00:40:11.160 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../007-python3-cephfs_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:11.200 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-cephfs (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:11.246 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package librgw2. 2021-03-11T00:40:11.259 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../013-librgw2_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:11.288 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking librgw2 (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:11.511 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-ceph-common. 2021-03-11T00:40:11.523 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../008-python3-ceph-common_15.2.9-108-g1ff7d244-1bionic_all.deb ... 2021-03-11T00:40:11.561 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-ceph-common (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:11.799 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-rgw. 2021-03-11T00:40:11.812 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../014-python3-rgw_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:11.838 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-prettytable. 2021-03-11T00:40:11.849 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-rgw (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:11.852 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../009-python3-prettytable_0.7.2-3_all.deb ... 2021-03-11T00:40:11.880 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2021-03-11T00:40:12.133 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-rbd. 2021-03-11T00:40:12.144 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../010-python3-rbd_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:12.177 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2021-03-11T00:40:12.182 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-rbd (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:12.190 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../015-libsnappy1v5_1.1.7-1_amd64.deb ... 2021-03-11T00:40:12.227 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2021-03-11T00:40:12.547 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2021-03-11T00:40:12.552 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package librabbitmq4:amd64. 2021-03-11T00:40:12.558 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../016-libleveldb1v5_1.20-2_amd64.deb ... 2021-03-11T00:40:12.563 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../011-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2021-03-11T00:40:12.596 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2021-03-11T00:40:12.601 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-03-11T00:40:12.915 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package liboath0. 2021-03-11T00:40:12.920 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package librdkafka1:amd64. 2021-03-11T00:40:12.926 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../017-liboath0_2.6.1-1_amd64.deb ... 2021-03-11T00:40:12.935 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../012-librdkafka1_0.11.3-1build1_amd64.deb ... 2021-03-11T00:40:12.957 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking liboath0 (2.6.1-1) ... 2021-03-11T00:40:12.970 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2021-03-11T00:40:13.264 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package librgw2. 2021-03-11T00:40:13.267 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libradosstriper1. 2021-03-11T00:40:13.277 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../013-librgw2_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:13.279 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../018-libradosstriper1_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:13.315 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking librgw2 (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:13.385 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libradosstriper1 (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:13.796 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-common. 2021-03-11T00:40:13.808 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../019-ceph-common_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:13.833 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-rgw. 2021-03-11T00:40:13.846 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-common (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:13.848 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../014-python3-rgw_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:13.883 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-rgw (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:14.202 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2021-03-11T00:40:14.214 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../015-libsnappy1v5_1.1.7-1_amd64.deb ... 2021-03-11T00:40:14.252 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2021-03-11T00:40:14.546 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2021-03-11T00:40:14.558 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../016-libleveldb1v5_1.20-2_amd64.deb ... 2021-03-11T00:40:14.600 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2021-03-11T00:40:14.941 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package liboath0. 2021-03-11T00:40:14.953 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../017-liboath0_2.6.1-1_amd64.deb ... 2021-03-11T00:40:14.991 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking liboath0 (2.6.1-1) ... 2021-03-11T00:40:15.302 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package libradosstriper1. 2021-03-11T00:40:15.314 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../018-libradosstriper1_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:15.352 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking libradosstriper1 (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:15.593 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-base. 2021-03-11T00:40:15.604 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../020-ceph-base_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:15.656 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-base (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:15.813 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package ceph-common. 2021-03-11T00:40:15.823 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../019-ceph-common_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:15.863 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking ceph-common (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:16.542 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-dateutil. 2021-03-11T00:40:16.551 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../021-python3-dateutil_2.6.1-1_all.deb ... 2021-03-11T00:40:16.589 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2021-03-11T00:40:16.933 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-asn1crypto. 2021-03-11T00:40:16.944 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../022-python3-asn1crypto_0.24.0-1_all.deb ... 2021-03-11T00:40:16.983 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2021-03-11T00:40:17.252 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-cffi-backend. 2021-03-11T00:40:17.265 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../023-python3-cffi-backend_1.11.5-1_amd64.deb ... 2021-03-11T00:40:17.302 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2021-03-11T00:40:17.601 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package ceph-base. 2021-03-11T00:40:17.617 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../020-ceph-base_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:17.628 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-cryptography. 2021-03-11T00:40:17.634 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../024-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2021-03-11T00:40:17.661 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking ceph-base (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:17.682 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-03-11T00:40:18.042 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-openssl. 2021-03-11T00:40:18.056 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../025-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2021-03-11T00:40:18.091 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2021-03-11T00:40:18.386 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2021-03-11T00:40:18.399 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../026-ceph-mgr-modules-core_15.2.9-108-g1ff7d244-1bionic_all.deb ... 2021-03-11T00:40:18.436 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mgr-modules-core (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:18.530 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-dateutil. 2021-03-11T00:40:18.541 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../021-python3-dateutil_2.6.1-1_all.deb ... 2021-03-11T00:40:18.583 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2021-03-11T00:40:18.783 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-bcrypt. 2021-03-11T00:40:18.797 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../027-python3-bcrypt_3.1.4-2_amd64.deb ... 2021-03-11T00:40:18.831 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2021-03-11T00:40:18.916 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-asn1crypto. 2021-03-11T00:40:18.927 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../022-python3-asn1crypto_0.24.0-1_all.deb ... 2021-03-11T00:40:18.970 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2021-03-11T00:40:19.125 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-cherrypy3. 2021-03-11T00:40:19.138 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-2_all.deb ... 2021-03-11T00:40:19.184 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2021-03-11T00:40:19.243 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-cffi-backend. 2021-03-11T00:40:19.251 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../023-python3-cffi-backend_1.11.5-1_amd64.deb ... 2021-03-11T00:40:19.294 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2021-03-11T00:40:19.512 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-jwt. 2021-03-11T00:40:19.524 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../029-python3-jwt_1.5.3+ds1-1_all.deb ... 2021-03-11T00:40:19.562 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2021-03-11T00:40:19.563 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-cryptography. 2021-03-11T00:40:19.575 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../024-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2021-03-11T00:40:19.612 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-03-11T00:40:19.889 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-logutils. 2021-03-11T00:40:19.904 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-5_all.deb ... 2021-03-11T00:40:19.940 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-logutils (0.3.3-5) ... 2021-03-11T00:40:19.983 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-openssl. 2021-03-11T00:40:19.994 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../025-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2021-03-11T00:40:20.032 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2021-03-11T00:40:20.201 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-markupsafe. 2021-03-11T00:40:20.213 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../031-python3-markupsafe_1.0-1build1_amd64.deb ... 2021-03-11T00:40:20.242 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2021-03-11T00:40:20.294 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2021-03-11T00:40:20.305 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../026-ceph-mgr-modules-core_15.2.9-108-g1ff7d244-1bionic_all.deb ... 2021-03-11T00:40:20.345 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking ceph-mgr-modules-core (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:20.528 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-mako. 2021-03-11T00:40:20.540 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../032-python3-mako_1.0.7+ds1-1_all.deb ... 2021-03-11T00:40:20.576 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2021-03-11T00:40:20.813 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-bcrypt. 2021-03-11T00:40:20.824 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../027-python3-bcrypt_3.1.4-2_amd64.deb ... 2021-03-11T00:40:20.863 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2021-03-11T00:40:20.898 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-simplegeneric. 2021-03-11T00:40:20.909 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-1_all.deb ... 2021-03-11T00:40:20.947 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2021-03-11T00:40:21.149 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-cherrypy3. 2021-03-11T00:40:21.162 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-2_all.deb ... 2021-03-11T00:40:21.200 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2021-03-11T00:40:21.258 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-singledispatch. 2021-03-11T00:40:21.265 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2_all.deb ... 2021-03-11T00:40:21.308 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2021-03-11T00:40:21.477 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-jwt. 2021-03-11T00:40:21.489 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../029-python3-jwt_1.5.3+ds1-1_all.deb ... 2021-03-11T00:40:21.519 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2021-03-11T00:40:21.627 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-webob. 2021-03-11T00:40:21.639 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../035-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2021-03-11T00:40:21.677 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2021-03-11T00:40:21.830 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-logutils. 2021-03-11T00:40:21.842 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-5_all.deb ... 2021-03-11T00:40:21.879 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-logutils (0.3.3-5) ... 2021-03-11T00:40:21.971 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-bs4. 2021-03-11T00:40:21.983 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../036-python3-bs4_4.6.0-1_all.deb ... 2021-03-11T00:40:22.013 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2021-03-11T00:40:22.191 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-markupsafe. 2021-03-11T00:40:22.203 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../031-python3-markupsafe_1.0-1build1_amd64.deb ... 2021-03-11T00:40:22.240 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2021-03-11T00:40:22.316 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-waitress. 2021-03-11T00:40:22.329 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../037-python3-waitress_1.0.1-1_all.deb ... 2021-03-11T00:40:22.366 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-waitress (1.0.1-1) ... 2021-03-11T00:40:22.527 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-mako. 2021-03-11T00:40:22.540 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../032-python3-mako_1.0.7+ds1-1_all.deb ... 2021-03-11T00:40:22.576 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2021-03-11T00:40:22.719 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-tempita. 2021-03-11T00:40:22.730 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../038-python3-tempita_0.5.2-2_all.deb ... 2021-03-11T00:40:22.768 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-tempita (0.5.2-2) ... 2021-03-11T00:40:22.894 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-simplegeneric. 2021-03-11T00:40:22.906 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-1_all.deb ... 2021-03-11T00:40:22.944 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2021-03-11T00:40:23.068 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-paste. 2021-03-11T00:40:23.081 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../039-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2021-03-11T00:40:23.117 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-03-11T00:40:23.246 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-singledispatch. 2021-03-11T00:40:23.259 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2_all.deb ... 2021-03-11T00:40:23.296 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2021-03-11T00:40:23.437 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2021-03-11T00:40:23.447 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../040-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2021-03-11T00:40:23.487 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2021-03-11T00:40:23.582 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-webob. 2021-03-11T00:40:23.594 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../035-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2021-03-11T00:40:23.632 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2021-03-11T00:40:23.799 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pastedeploy. 2021-03-11T00:40:23.808 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../041-python3-pastedeploy_1.5.2-4_all.deb ... 2021-03-11T00:40:23.848 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2021-03-11T00:40:23.928 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-bs4. 2021-03-11T00:40:23.939 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../036-python3-bs4_4.6.0-1_all.deb ... 2021-03-11T00:40:23.977 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2021-03-11T00:40:24.142 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-webtest. 2021-03-11T00:40:24.154 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../042-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2021-03-11T00:40:24.217 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2021-03-11T00:40:24.270 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-waitress. 2021-03-11T00:40:24.283 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../037-python3-waitress_1.0.1-1_all.deb ... 2021-03-11T00:40:24.320 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-waitress (1.0.1-1) ... 2021-03-11T00:40:24.553 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pecan. 2021-03-11T00:40:24.564 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../043-python3-pecan_1.2.1-2_all.deb ... 2021-03-11T00:40:24.603 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pecan (1.2.1-2) ... 2021-03-11T00:40:24.665 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-tempita. 2021-03-11T00:40:24.678 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../038-python3-tempita_0.5.2-2_all.deb ... 2021-03-11T00:40:24.715 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-tempita (0.5.2-2) ... 2021-03-11T00:40:24.949 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-werkzeug. 2021-03-11T00:40:24.962 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../044-python3-werkzeug_0.14.1+dfsg1-1ubuntu0.1_all.deb ... 2021-03-11T00:40:25.000 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-03-11T00:40:25.010 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-paste. 2021-03-11T00:40:25.026 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../039-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2021-03-11T00:40:25.060 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-03-11T00:40:25.371 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mgr. 2021-03-11T00:40:25.380 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2021-03-11T00:40:25.383 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../045-ceph-mgr_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:25.393 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../040-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2021-03-11T00:40:25.421 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mgr (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:25.430 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2021-03-11T00:40:25.700 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-pastedeploy. 2021-03-11T00:40:25.713 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../041-python3-pastedeploy_1.5.2-4_all.deb ... 2021-03-11T00:40:25.741 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2021-03-11T00:40:25.869 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mon. 2021-03-11T00:40:25.881 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../046-ceph-mon_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:25.918 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mon (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:26.077 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-webtest. 2021-03-11T00:40:26.088 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../042-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2021-03-11T00:40:26.127 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2021-03-11T00:40:26.477 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-pecan. 2021-03-11T00:40:26.485 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../043-python3-pecan_1.2.1-2_all.deb ... 2021-03-11T00:40:26.522 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-pecan (1.2.1-2) ... 2021-03-11T00:40:26.822 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-osd. 2021-03-11T00:40:26.834 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../047-ceph-osd_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:26.868 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-werkzeug. 2021-03-11T00:40:26.871 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-osd (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:26.881 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../044-python3-werkzeug_0.14.1+dfsg1-1ubuntu0.1_all.deb ... 2021-03-11T00:40:26.918 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-03-11T00:40:27.288 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package ceph-mgr. 2021-03-11T00:40:27.299 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../045-ceph-mgr_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:27.338 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking ceph-mgr (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:27.791 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package ceph-mon. 2021-03-11T00:40:27.803 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../046-ceph-mon_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:27.841 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking ceph-mon (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:28.712 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package ceph-osd. 2021-03-11T00:40:28.724 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../047-ceph-osd_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:28.761 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking ceph-osd (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:28.785 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph. 2021-03-11T00:40:28.796 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../048-ceph_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:28.835 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:29.154 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-fuse. 2021-03-11T00:40:29.165 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../049-ceph-fuse_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:29.204 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-fuse (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:30.547 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mds. 2021-03-11T00:40:30.558 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../050-ceph-mds_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:30.597 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mds (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:31.066 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package cephadm. 2021-03-11T00:40:31.066 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package ceph. 2021-03-11T00:40:31.078 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../051-cephadm_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:31.079 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../048-ceph_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:31.108 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking ceph (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:31.116 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking cephadm (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:31.411 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package ceph-fuse. 2021-03-11T00:40:31.422 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../049-ceph-fuse_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:31.427 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-jinja2. 2021-03-11T00:40:31.439 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../052-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2021-03-11T00:40:31.461 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking ceph-fuse (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:31.477 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-03-11T00:40:31.816 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2021-03-11T00:40:31.824 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../053-ceph-mgr-cephadm_15.2.9-108-g1ff7d244-1bionic_all.deb ... 2021-03-11T00:40:31.849 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package ceph-mds. 2021-03-11T00:40:31.861 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../050-ceph-mds_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:31.863 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mgr-cephadm (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:31.899 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking ceph-mds (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:32.191 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-repoze.lru. 2021-03-11T00:40:32.204 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../054-python3-repoze.lru_0.7-1_all.deb ... 2021-03-11T00:40:32.242 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2021-03-11T00:40:32.385 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package cephadm. 2021-03-11T00:40:32.398 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../051-cephadm_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:32.435 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking cephadm (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:32.511 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-routes. 2021-03-11T00:40:32.521 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../055-python3-routes_2.4.1-1_all.deb ... 2021-03-11T00:40:32.552 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-routes (2.4.1-1) ... 2021-03-11T00:40:32.741 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-jinja2. 2021-03-11T00:40:32.751 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../052-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2021-03-11T00:40:32.788 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-03-11T00:40:32.872 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2021-03-11T00:40:32.884 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../056-ceph-mgr-dashboard_15.2.9-108-g1ff7d244-1bionic_all.deb ... 2021-03-11T00:40:32.922 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mgr-dashboard (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:33.117 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2021-03-11T00:40:33.130 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../053-ceph-mgr-cephadm_15.2.9-108-g1ff7d244-1bionic_all.deb ... 2021-03-11T00:40:33.167 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking ceph-mgr-cephadm (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:33.479 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-repoze.lru. 2021-03-11T00:40:33.492 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../054-python3-repoze.lru_0.7-1_all.deb ... 2021-03-11T00:40:33.528 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2021-03-11T00:40:33.807 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-routes. 2021-03-11T00:40:33.820 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../055-python3-routes_2.4.1-1_all.deb ... 2021-03-11T00:40:33.857 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-routes (2.4.1-1) ... 2021-03-11T00:40:33.865 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2021-03-11T00:40:33.879 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../057-ceph-mgr-diskprediction-cloud_15.2.9-108-g1ff7d244-1bionic_all.deb ... 2021-03-11T00:40:33.915 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:34.219 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-numpy. 2021-03-11T00:40:34.229 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../058-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2021-03-11T00:40:34.260 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2021-03-11T00:40:34.268 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2021-03-11T00:40:34.272 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../056-ceph-mgr-dashboard_15.2.9-108-g1ff7d244-1bionic_all.deb ... 2021-03-11T00:40:34.310 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking ceph-mgr-dashboard (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:34.849 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-decorator. 2021-03-11T00:40:34.862 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../059-python3-decorator_4.1.2-1_all.deb ... 2021-03-11T00:40:34.924 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-decorator (4.1.2-1) ... 2021-03-11T00:40:35.218 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-scipy. 2021-03-11T00:40:35.222 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2021-03-11T00:40:35.233 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../060-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2021-03-11T00:40:35.236 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../057-ceph-mgr-diskprediction-cloud_15.2.9-108-g1ff7d244-1bionic_all.deb ... 2021-03-11T00:40:35.268 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2021-03-11T00:40:35.297 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:35.601 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-numpy. 2021-03-11T00:40:35.613 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../058-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2021-03-11T00:40:35.651 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2021-03-11T00:40:36.265 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-decorator. 2021-03-11T00:40:36.283 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../059-python3-decorator_4.1.2-1_all.deb ... 2021-03-11T00:40:36.315 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-decorator (4.1.2-1) ... 2021-03-11T00:40:36.384 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-sklearn-lib. 2021-03-11T00:40:36.396 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../061-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2021-03-11T00:40:36.452 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2021-03-11T00:40:36.618 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-scipy. 2021-03-11T00:40:36.630 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../060-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2021-03-11T00:40:36.668 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2021-03-11T00:40:36.905 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-joblib. 2021-03-11T00:40:36.916 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../062-python3-joblib_0.11-1_all.deb ... 2021-03-11T00:40:36.954 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-joblib (0.11-1) ... 2021-03-11T00:40:37.316 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-sklearn. 2021-03-11T00:40:37.329 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../063-python3-sklearn_0.19.1-3_all.deb ... 2021-03-11T00:40:37.366 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2021-03-11T00:40:37.744 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-sklearn-lib. 2021-03-11T00:40:37.756 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../061-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2021-03-11T00:40:37.794 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2021-03-11T00:40:37.915 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2021-03-11T00:40:37.932 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../064-ceph-mgr-diskprediction-local_15.2.9-108-g1ff7d244-1bionic_all.deb ... 2021-03-11T00:40:37.964 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:38.248 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-joblib. 2021-03-11T00:40:38.259 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../062-python3-joblib_0.11-1_all.deb ... 2021-03-11T00:40:38.289 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-joblib (0.11-1) ... 2021-03-11T00:40:38.668 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-sklearn. 2021-03-11T00:40:38.679 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../063-python3-sklearn_0.19.1-3_all.deb ... 2021-03-11T00:40:38.718 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2021-03-11T00:40:39.093 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-monotonic. 2021-03-11T00:40:39.104 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../065-python3-monotonic_1.1-2_all.deb ... 2021-03-11T00:40:39.143 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-monotonic (1.1-2) ... 2021-03-11T00:40:39.293 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2021-03-11T00:40:39.307 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../064-ceph-mgr-diskprediction-local_15.2.9-108-g1ff7d244-1bionic_all.deb ... 2021-03-11T00:40:39.343 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:39.482 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-fasteners. 2021-03-11T00:40:39.491 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../066-python3-fasteners_0.12.0-3_all.deb ... 2021-03-11T00:40:39.534 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2021-03-11T00:40:39.824 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-httplib2. 2021-03-11T00:40:39.844 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../067-python3-httplib2_0.9.2+dfsg-1ubuntu0.3_all.deb ... 2021-03-11T00:40:39.882 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.3) ... 2021-03-11T00:40:40.126 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pyasn1. 2021-03-11T00:40:40.139 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../068-python3-pyasn1_0.4.2-3_all.deb ... 2021-03-11T00:40:40.210 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2021-03-11T00:40:40.580 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pyasn1-modules. 2021-03-11T00:40:40.588 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-monotonic. 2021-03-11T00:40:40.593 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../069-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2021-03-11T00:40:40.603 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../065-python3-monotonic_1.1-2_all.deb ... 2021-03-11T00:40:40.638 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-monotonic (1.1-2) ... 2021-03-11T00:40:40.639 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2021-03-11T00:40:40.941 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-fasteners. 2021-03-11T00:40:40.954 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../066-python3-fasteners_0.12.0-3_all.deb ... 2021-03-11T00:40:40.983 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-rsa. 2021-03-11T00:40:40.991 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2021-03-11T00:40:40.995 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../070-python3-rsa_3.4.2-1_all.deb ... 2021-03-11T00:40:41.032 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-rsa (3.4.2-1) ... 2021-03-11T00:40:41.294 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-httplib2. 2021-03-11T00:40:41.306 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../067-python3-httplib2_0.9.2+dfsg-1ubuntu0.3_all.deb ... 2021-03-11T00:40:41.337 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.3) ... 2021-03-11T00:40:41.344 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-sqlalchemy. 2021-03-11T00:40:41.356 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../071-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2021-03-11T00:40:41.394 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-03-11T00:40:41.656 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-pyasn1. 2021-03-11T00:40:41.668 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../068-python3-pyasn1_0.4.2-3_all.deb ... 2021-03-11T00:40:41.708 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2021-03-11T00:40:41.840 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-oauth2client. 2021-03-11T00:40:41.854 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../072-python3-oauth2client_4.1.2-3_all.deb ... 2021-03-11T00:40:41.890 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2021-03-11T00:40:42.018 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-pyasn1-modules. 2021-03-11T00:40:42.032 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../069-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2021-03-11T00:40:42.077 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2021-03-11T00:40:42.218 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-setuptools. 2021-03-11T00:40:42.230 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../073-python3-setuptools_39.0.1-2_all.deb ... 2021-03-11T00:40:42.268 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2021-03-11T00:40:42.347 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-rsa. 2021-03-11T00:40:42.359 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../070-python3-rsa_3.4.2-1_all.deb ... 2021-03-11T00:40:42.396 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-rsa (3.4.2-1) ... 2021-03-11T00:40:42.671 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-websocket. 2021-03-11T00:40:42.685 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../074-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2021-03-11T00:40:42.708 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-sqlalchemy. 2021-03-11T00:40:42.719 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../071-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2021-03-11T00:40:42.721 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2021-03-11T00:40:42.758 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-03-11T00:40:43.033 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-kubernetes. 2021-03-11T00:40:43.044 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../075-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2021-03-11T00:40:43.083 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2021-03-11T00:40:43.205 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-oauth2client. 2021-03-11T00:40:43.217 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../072-python3-oauth2client_4.1.2-3_all.deb ... 2021-03-11T00:40:43.255 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2021-03-11T00:40:43.503 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2021-03-11T00:40:43.516 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.9-108-g1ff7d244-1bionic_all.deb ... 2021-03-11T00:40:43.553 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mgr-k8sevents (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:43.566 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-setuptools. 2021-03-11T00:40:43.580 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../073-python3-setuptools_39.0.1-2_all.deb ... 2021-03-11T00:40:43.626 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2021-03-11T00:40:43.848 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-json-pointer. 2021-03-11T00:40:43.860 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../077-python3-json-pointer_1.10-1_all.deb ... 2021-03-11T00:40:43.898 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-json-pointer (1.10-1) ... 2021-03-11T00:40:44.045 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-websocket. 2021-03-11T00:40:44.058 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../074-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2021-03-11T00:40:44.095 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2021-03-11T00:40:44.269 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-jsonpatch. 2021-03-11T00:40:44.280 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../078-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2021-03-11T00:40:44.320 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-03-11T00:40:44.407 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-kubernetes. 2021-03-11T00:40:44.421 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../075-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2021-03-11T00:40:44.490 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2021-03-11T00:40:44.620 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mgr-rook. 2021-03-11T00:40:44.634 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../079-ceph-mgr-rook_15.2.9-108-g1ff7d244-1bionic_all.deb ... 2021-03-11T00:40:44.687 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mgr-rook (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:44.878 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2021-03-11T00:40:44.891 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.9-108-g1ff7d244-1bionic_all.deb ... 2021-03-11T00:40:44.937 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking ceph-mgr-k8sevents (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:44.973 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libonig4:amd64. 2021-03-11T00:40:44.985 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../080-libonig4_6.7.0-1_amd64.deb ... 2021-03-11T00:40:45.091 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2021-03-11T00:40:45.256 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-json-pointer. 2021-03-11T00:40:45.268 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../077-python3-json-pointer_1.10-1_all.deb ... 2021-03-11T00:40:45.307 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-json-pointer (1.10-1) ... 2021-03-11T00:40:45.536 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libjq1:amd64. 2021-03-11T00:40:45.548 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../081-libjq1_1.5+dfsg-2_amd64.deb ... 2021-03-11T00:40:45.586 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2021-03-11T00:40:45.661 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-jsonpatch. 2021-03-11T00:40:45.671 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../078-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2021-03-11T00:40:45.713 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-03-11T00:40:45.864 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package jq. 2021-03-11T00:40:45.877 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../082-jq_1.5+dfsg-2_amd64.deb ... 2021-03-11T00:40:45.914 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking jq (1.5+dfsg-2) ... 2021-03-11T00:40:46.013 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package ceph-mgr-rook. 2021-03-11T00:40:46.026 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../079-ceph-mgr-rook_15.2.9-108-g1ff7d244-1bionic_all.deb ... 2021-03-11T00:40:46.063 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking ceph-mgr-rook (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:46.141 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package socat. 2021-03-11T00:40:46.154 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../083-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2021-03-11T00:40:46.191 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2021-03-11T00:40:46.392 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package libonig4:amd64. 2021-03-11T00:40:46.405 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../080-libonig4_6.7.0-1_amd64.deb ... 2021-03-11T00:40:46.522 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2021-03-11T00:40:46.586 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libxslt1.1:amd64. 2021-03-11T00:40:46.599 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../084-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2021-03-11T00:40:46.636 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-03-11T00:40:46.863 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package libjq1:amd64. 2021-03-11T00:40:46.867 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../081-libjq1_1.5+dfsg-2_amd64.deb ... 2021-03-11T00:40:46.905 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2021-03-11T00:40:46.907 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package xmlstarlet. 2021-03-11T00:40:46.918 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../085-xmlstarlet_1.6.1-2_amd64.deb ... 2021-03-11T00:40:46.982 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2021-03-11T00:40:47.166 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package jq. 2021-03-11T00:40:47.180 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../082-jq_1.5+dfsg-2_amd64.deb ... 2021-03-11T00:40:47.217 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking jq (1.5+dfsg-2) ... 2021-03-11T00:40:47.310 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-test. 2021-03-11T00:40:47.323 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../086-ceph-test_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:47.360 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-test (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:47.469 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package socat. 2021-03-11T00:40:47.482 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../083-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2021-03-11T00:40:47.519 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2021-03-11T00:40:47.890 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package libxslt1.1:amd64. 2021-03-11T00:40:47.902 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../084-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2021-03-11T00:40:47.940 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-03-11T00:40:48.218 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package xmlstarlet. 2021-03-11T00:40:48.231 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../085-xmlstarlet_1.6.1-2_amd64.deb ... 2021-03-11T00:40:48.268 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2021-03-11T00:40:48.607 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package ceph-test. 2021-03-11T00:40:48.619 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../086-ceph-test_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:48.656 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking ceph-test (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:50.672 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libcephfs-dev. 2021-03-11T00:40:50.685 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../087-libcephfs-dev_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:50.730 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libcephfs-dev (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:51.006 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package nvme-cli. 2021-03-11T00:40:51.011 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../088-nvme-cli_1.5-1ubuntu1.1_amd64.deb ... 2021-03-11T00:40:51.050 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking nvme-cli (1.5-1ubuntu1.1) ... 2021-03-11T00:40:51.370 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python-attr. 2021-03-11T00:40:51.384 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../089-python-attr_17.4.0-2_all.deb ... 2021-03-11T00:40:51.453 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python-attr (17.4.0-2) ... 2021-03-11T00:40:51.749 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python-funcsigs. 2021-03-11T00:40:51.762 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../090-python-funcsigs_1.0.2-4_all.deb ... 2021-03-11T00:40:51.806 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2021-03-11T00:40:52.002 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package libcephfs-dev. 2021-03-11T00:40:52.013 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../087-libcephfs-dev_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:40:52.052 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking libcephfs-dev (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:40:52.076 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python-pluggy. 2021-03-11T00:40:52.088 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../091-python-pluggy_0.6.0-1_all.deb ... 2021-03-11T00:40:52.117 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python-pluggy (0.6.0-1) ... 2021-03-11T00:40:52.322 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package nvme-cli. 2021-03-11T00:40:52.334 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../088-nvme-cli_1.5-1ubuntu1.1_amd64.deb ... 2021-03-11T00:40:52.372 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking nvme-cli (1.5-1ubuntu1.1) ... 2021-03-11T00:40:52.420 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python-py. 2021-03-11T00:40:52.434 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../092-python-py_1.5.2-1_all.deb ... 2021-03-11T00:40:52.479 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python-py (1.5.2-1) ... 2021-03-11T00:40:52.718 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python-attr. 2021-03-11T00:40:52.730 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../089-python-attr_17.4.0-2_all.deb ... 2021-03-11T00:40:52.768 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python-attr (17.4.0-2) ... 2021-03-11T00:40:52.799 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python-pytest. 2021-03-11T00:40:52.811 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../093-python-pytest_3.3.2-2_all.deb ... 2021-03-11T00:40:52.848 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python-pytest (3.3.2-2) ... 2021-03-11T00:40:53.080 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python-funcsigs. 2021-03-11T00:40:53.094 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../090-python-funcsigs_1.0.2-4_all.deb ... 2021-03-11T00:40:53.139 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2021-03-11T00:40:53.210 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-attr. 2021-03-11T00:40:53.223 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../094-python3-attr_17.4.0-2_all.deb ... 2021-03-11T00:40:53.260 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-attr (17.4.0-2) ... 2021-03-11T00:40:53.433 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python-pluggy. 2021-03-11T00:40:53.444 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../091-python-pluggy_0.6.0-1_all.deb ... 2021-03-11T00:40:53.484 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python-pluggy (0.6.0-1) ... 2021-03-11T00:40:53.563 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-cycler. 2021-03-11T00:40:53.581 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../095-python3-cycler_0.10.0-1_all.deb ... 2021-03-11T00:40:53.614 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-cycler (0.10.0-1) ... 2021-03-11T00:40:53.794 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python-py. 2021-03-11T00:40:53.806 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../092-python-py_1.5.2-1_all.deb ... 2021-03-11T00:40:53.844 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python-py (1.5.2-1) ... 2021-03-11T00:40:53.891 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-webencodings. 2021-03-11T00:40:53.907 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../096-python3-webencodings_0.5-2_all.deb ... 2021-03-11T00:40:53.933 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-webencodings (0.5-2) ... 2021-03-11T00:40:54.165 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python-pytest. 2021-03-11T00:40:54.177 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../093-python-pytest_3.3.2-2_all.deb ... 2021-03-11T00:40:54.215 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python-pytest (3.3.2-2) ... 2021-03-11T00:40:54.219 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-html5lib. 2021-03-11T00:40:54.232 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../097-python3-html5lib_0.999999999-1_all.deb ... 2021-03-11T00:40:54.268 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2021-03-11T00:40:54.535 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-attr. 2021-03-11T00:40:54.548 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../094-python3-attr_17.4.0-2_all.deb ... 2021-03-11T00:40:54.585 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-attr (17.4.0-2) ... 2021-03-11T00:40:54.638 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-lxml:amd64. 2021-03-11T00:40:54.652 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../098-python3-lxml_4.2.1-1ubuntu0.3_amd64.deb ... 2021-03-11T00:40:54.697 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.3) ... 2021-03-11T00:40:54.855 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-cycler. 2021-03-11T00:40:54.866 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../095-python3-cycler_0.10.0-1_all.deb ... 2021-03-11T00:40:54.897 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-cycler (0.10.0-1) ... 2021-03-11T00:40:55.133 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pyparsing. 2021-03-11T00:40:55.146 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../099-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2021-03-11T00:40:55.183 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2021-03-11T00:40:55.184 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-webencodings. 2021-03-11T00:40:55.194 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../096-python3-webencodings_0.5-2_all.deb ... 2021-03-11T00:40:55.258 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-webencodings (0.5-2) ... 2021-03-11T00:40:55.511 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-tz. 2021-03-11T00:40:55.524 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../100-python3-tz_2018.3-2_all.deb ... 2021-03-11T00:40:55.566 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-html5lib. 2021-03-11T00:40:55.568 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-tz (2018.3-2) ... 2021-03-11T00:40:55.575 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../097-python3-html5lib_0.999999999-1_all.deb ... 2021-03-11T00:40:55.611 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2021-03-11T00:40:55.864 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-matplotlib. 2021-03-11T00:40:55.910 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../101-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2021-03-11T00:40:55.914 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2021-03-11T00:40:55.972 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-lxml:amd64. 2021-03-11T00:40:55.987 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../098-python3-lxml_4.2.1-1ubuntu0.3_amd64.deb ... 2021-03-11T00:40:56.015 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.3) ... 2021-03-11T00:40:56.452 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-pyparsing. 2021-03-11T00:40:56.463 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../099-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2021-03-11T00:40:56.502 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2021-03-11T00:40:56.569 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-olefile. 2021-03-11T00:40:56.582 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../102-python3-olefile_0.45.1-1_all.deb ... 2021-03-11T00:40:56.611 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-olefile (0.45.1-1) ... 2021-03-11T00:40:56.839 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-tz. 2021-03-11T00:40:56.850 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../100-python3-tz_2018.3-2_all.deb ... 2021-03-11T00:40:56.889 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-tz (2018.3-2) ... 2021-03-11T00:40:56.931 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pastescript. 2021-03-11T00:40:56.942 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-2_all.deb ... 2021-03-11T00:40:56.981 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2021-03-11T00:40:57.192 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-matplotlib. 2021-03-11T00:40:57.204 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../101-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2021-03-11T00:40:57.241 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2021-03-11T00:40:57.302 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pil:amd64. 2021-03-11T00:40:57.312 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../104-python3-pil_5.1.0-1ubuntu0.4_amd64.deb ... 2021-03-11T00:40:57.350 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.4) ... 2021-03-11T00:40:57.737 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pluggy. 2021-03-11T00:40:57.748 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../105-python3-pluggy_0.6.0-1_all.deb ... 2021-03-11T00:40:57.803 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2021-03-11T00:40:57.898 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-olefile. 2021-03-11T00:40:57.912 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../102-python3-olefile_0.45.1-1_all.deb ... 2021-03-11T00:40:57.957 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-olefile (0.45.1-1) ... 2021-03-11T00:40:58.090 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-py. 2021-03-11T00:40:58.104 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../106-python3-py_1.5.2-1_all.deb ... 2021-03-11T00:40:58.139 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-py (1.5.2-1) ... 2021-03-11T00:40:58.259 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-pastescript. 2021-03-11T00:40:58.271 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-2_all.deb ... 2021-03-11T00:40:58.309 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2021-03-11T00:40:58.459 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pyinotify. 2021-03-11T00:40:58.472 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1_all.deb ... 2021-03-11T00:40:58.501 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2021-03-11T00:40:58.621 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-pil:amd64. 2021-03-11T00:40:58.639 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../104-python3-pil_5.1.0-1ubuntu0.4_amd64.deb ... 2021-03-11T00:40:58.672 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.4) ... 2021-03-11T00:40:58.795 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pytest. 2021-03-11T00:40:58.808 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../108-python3-pytest_3.3.2-2_all.deb ... 2021-03-11T00:40:58.848 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pytest (3.3.2-2) ... 2021-03-11T00:40:59.042 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-pluggy. 2021-03-11T00:40:59.054 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../105-python3-pluggy_0.6.0-1_all.deb ... 2021-03-11T00:40:59.093 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2021-03-11T00:40:59.182 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-simplejson. 2021-03-11T00:40:59.196 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../109-python3-simplejson_3.13.2-1_amd64.deb ... 2021-03-11T00:40:59.240 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2021-03-11T00:40:59.416 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-py. 2021-03-11T00:40:59.425 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../106-python3-py_1.5.2-1_all.deb ... 2021-03-11T00:40:59.447 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-py (1.5.2-1) ... 2021-03-11T00:40:59.579 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-tk:amd64. 2021-03-11T00:40:59.590 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../110-python3-tk_3.6.9-1~18.04_amd64.deb ... 2021-03-11T00:40:59.637 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2021-03-11T00:40:59.766 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-pyinotify. 2021-03-11T00:40:59.787 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1_all.deb ... 2021-03-11T00:40:59.836 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2021-03-11T00:41:00.024 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package radosgw. 2021-03-11T00:41:00.037 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../111-radosgw_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:41:00.074 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking radosgw (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:00.135 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-pytest. 2021-03-11T00:41:00.147 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../108-python3-pytest_3.3.2-2_all.deb ... 2021-03-11T00:41:00.194 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-pytest (3.3.2-2) ... 2021-03-11T00:41:00.517 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-simplejson. 2021-03-11T00:41:00.528 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../109-python3-simplejson_3.13.2-1_amd64.deb ... 2021-03-11T00:41:00.565 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2021-03-11T00:41:00.889 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package python3-tk:amd64. 2021-03-11T00:41:00.902 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../110-python3-tk_3.6.9-1~18.04_amd64.deb ... 2021-03-11T00:41:00.945 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2021-03-11T00:41:01.211 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package rbd-fuse. 2021-03-11T00:41:01.222 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../112-rbd-fuse_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:41:01.262 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking rbd-fuse (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:01.422 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package radosgw. 2021-03-11T00:41:01.434 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../111-radosgw_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:41:01.471 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking radosgw (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:01.528 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-logutils (0.3.3-5) ... 2021-03-11T00:41:01.849 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-03-11T00:41:02.217 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2021-03-11T00:41:02.343 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-03-11T00:41:02.444 INFO:teuthology.orchestra.run.smithi124.stdout:Selecting previously unselected package rbd-fuse. 2021-03-11T00:41:02.457 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing to unpack .../112-rbd-fuse_15.2.9-108-g1ff7d244-1bionic_amd64.deb ... 2021-03-11T00:41:02.494 INFO:teuthology.orchestra.run.smithi124.stdout:Unpacking rbd-fuse (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:02.792 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-logutils (0.3.3-5) ... 2021-03-11T00:41:02.962 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.4) ... 2021-03-11T00:41:03.082 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-03-11T00:41:03.330 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2021-03-11T00:41:03.433 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2021-03-11T00:41:03.559 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-03-11T00:41:03.589 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-webencodings (0.5-2) ... 2021-03-11T00:41:03.874 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python-attr (17.4.0-2) ... 2021-03-11T00:41:04.075 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2021-03-11T00:41:04.162 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.4) ... 2021-03-11T00:41:04.201 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-py (1.5.2-1) ... 2021-03-11T00:41:04.544 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-waitress (1.0.1-1) ... 2021-03-11T00:41:04.546 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2021-03-11T00:41:04.823 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-webencodings (0.5-2) ... 2021-03-11T00:41:04.939 INFO:teuthology.orchestra.run.smithi119.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2021-03-11T00:41:05.116 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python-attr (17.4.0-2) ... 2021-03-11T00:41:05.238 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-olefile (0.45.1-1) ... 2021-03-11T00:41:05.317 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2021-03-11T00:41:05.443 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-py (1.5.2-1) ... 2021-03-11T00:41:05.523 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-ceph-argparse (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:05.786 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-waitress (1.0.1-1) ... 2021-03-11T00:41:05.810 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-tempita (0.5.2-2) ... 2021-03-11T00:41:06.108 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2021-03-11T00:41:06.183 INFO:teuthology.orchestra.run.smithi124.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2021-03-11T00:41:06.234 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2021-03-11T00:41:06.489 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-olefile (0.45.1-1) ... 2021-03-11T00:41:06.790 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-ceph-argparse (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:07.091 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-tempita (0.5.2-2) ... 2021-03-11T00:41:07.170 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2021-03-11T00:41:07.392 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2021-03-11T00:41:07.464 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2021-03-11T00:41:07.523 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2021-03-11T00:41:07.723 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python-py (1.5.2-1) ... 2021-03-11T00:41:07.924 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-simplejson (3.13.2-1) ... 2021-03-11T00:41:08.284 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-json-pointer (1.10-1) ... 2021-03-11T00:41:08.373 INFO:teuthology.orchestra.run.smithi119.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2021-03-11T00:41:08.438 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2021-03-11T00:41:08.627 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2021-03-11T00:41:08.752 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python-pluggy (0.6.0-1) ... 2021-03-11T00:41:08.764 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2021-03-11T00:41:08.954 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2021-03-11T00:41:09.074 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python-py (1.5.2-1) ... 2021-03-11T00:41:09.229 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2021-03-11T00:41:09.300 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-simplejson (3.13.2-1) ... 2021-03-11T00:41:09.564 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-cycler (0.10.0-1) ... 2021-03-11T00:41:09.651 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-json-pointer (1.10-1) ... 2021-03-11T00:41:09.741 INFO:teuthology.orchestra.run.smithi124.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2021-03-11T00:41:09.848 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2021-03-11T00:41:09.986 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2021-03-11T00:41:10.112 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python-pluggy (0.6.0-1) ... 2021-03-11T00:41:10.200 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-bs4 (4.6.0-1) ... 2021-03-11T00:41:10.313 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2021-03-11T00:41:10.542 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pluggy (0.6.0-1) ... 2021-03-11T00:41:10.802 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2021-03-11T00:41:10.835 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up cephadm (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:11.265 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-cycler (0.10.0-1) ... 2021-03-11T00:41:11.544 INFO:teuthology.orchestra.run.smithi119.stdout:Adding system user cephadm....done 2021-03-11T00:41:11.550 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2021-03-11T00:41:11.700 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2021-03-11T00:41:11.926 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-bs4 (4.6.0-1) ... 2021-03-11T00:41:12.001 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-03-11T00:41:12.127 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up nvme-cli (1.5-1ubuntu1.1) ... 2021-03-11T00:41:12.236 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-pluggy (0.6.0-1) ... 2021-03-11T00:41:12.270 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2021-03-11T00:41:12.537 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up cephadm (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:12.621 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-prettytable (0.7.2-3) ... 2021-03-11T00:41:12.905 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2021-03-11T00:41:13.014 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2021-03-11T00:41:13.196 INFO:teuthology.orchestra.run.smithi124.stdout:Adding system user cephadm....done 2021-03-11T00:41:13.342 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-ceph-common (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:13.360 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2021-03-11T00:41:13.650 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-monotonic (1.1-2) ... 2021-03-11T00:41:13.662 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-03-11T00:41:13.788 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up nvme-cli (1.5-1ubuntu1.1) ... 2021-03-11T00:41:13.930 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2021-03-11T00:41:13.951 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2021-03-11T00:41:14.094 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-03-11T00:41:14.282 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-prettytable (0.7.2-3) ... 2021-03-11T00:41:14.453 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-html5lib (0.999999999-1) ... 2021-03-11T00:41:14.578 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2021-03-11T00:41:14.692 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2021-03-11T00:41:14.822 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-03-11T00:41:14.894 INFO:teuthology.orchestra.run.smithi119.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2021-03-11T00:41:14.927 INFO:teuthology.orchestra.run.smithi119.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2021-03-11T00:41:15.032 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-ceph-common (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:15.157 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-repoze.lru (0.7-1) ... 2021-03-11T00:41:15.328 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-monotonic (1.1-2) ... 2021-03-11T00:41:15.450 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.3) ... 2021-03-11T00:41:15.629 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2021-03-11T00:41:15.757 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-03-11T00:41:15.767 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2021-03-11T00:41:16.100 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-html5lib (0.999999999-1) ... 2021-03-11T00:41:16.225 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-attr (17.4.0-2) ... 2021-03-11T00:41:16.411 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python-funcsigs (1.0.2-4) ... 2021-03-11T00:41:16.467 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-03-11T00:41:16.556 INFO:teuthology.orchestra.run.smithi124.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2021-03-11T00:41:16.589 INFO:teuthology.orchestra.run.smithi124.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2021-03-11T00:41:16.604 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-03-11T00:41:16.810 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-repoze.lru (0.7-1) ... 2021-03-11T00:41:17.005 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2021-03-11T00:41:17.103 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.3) ... 2021-03-11T00:41:17.104 INFO:teuthology.orchestra.run.smithi119.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2021-03-11T00:41:17.357 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up liboath0 (2.6.1-1) ... 2021-03-11T00:41:17.397 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2021-03-11T00:41:17.525 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-03-11T00:41:17.651 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2021-03-11T00:41:17.722 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-attr (17.4.0-2) ... 2021-03-11T00:41:17.919 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2021-03-11T00:41:18.031 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python-funcsigs (1.0.2-4) ... 2021-03-11T00:41:18.045 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-setuptools (39.0.1-2) ... 2021-03-11T00:41:18.225 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-03-11T00:41:18.430 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2021-03-11T00:41:18.558 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-decorator (4.1.2-1) ... 2021-03-11T00:41:18.651 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2021-03-11T00:41:18.741 INFO:teuthology.orchestra.run.smithi124.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2021-03-11T00:41:18.824 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-tz (2018.3-2) ... 2021-03-11T00:41:18.994 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up liboath0 (2.6.1-1) ... 2021-03-11T00:41:19.116 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2021-03-11T00:41:19.171 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-03-11T00:41:19.297 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2021-03-11T00:41:19.409 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2021-03-11T00:41:19.548 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2021-03-11T00:41:19.674 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-setuptools (39.0.1-2) ... 2021-03-11T00:41:19.702 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-joblib (0.11-1) ... 2021-03-11T00:41:20.061 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2021-03-11T00:41:20.078 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-routes (2.4.1-1) ... 2021-03-11T00:41:20.185 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-decorator (4.1.2-1) ... 2021-03-11T00:41:20.379 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2021-03-11T00:41:20.469 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-tz (2018.3-2) ... 2021-03-11T00:41:20.688 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-dateutil (2.6.1-1) ... 2021-03-11T00:41:20.771 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2021-03-11T00:41:20.991 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-03-11T00:41:21.081 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2021-03-11T00:41:21.327 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-fasteners (0.12.0-3) ... 2021-03-11T00:41:21.381 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-joblib (0.11-1) ... 2021-03-11T00:41:21.621 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up jq (1.5+dfsg-2) ... 2021-03-11T00:41:21.744 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2021-03-11T00:41:21.749 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-routes (2.4.1-1) ... 2021-03-11T00:41:22.034 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2021-03-11T00:41:22.336 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-dateutil (2.6.1-1) ... 2021-03-11T00:41:22.659 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-03-11T00:41:22.898 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.3) ... 2021-03-11T00:41:23.029 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-fasteners (0.12.0-3) ... 2021-03-11T00:41:23.200 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2021-03-11T00:41:23.343 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up jq (1.5+dfsg-2) ... 2021-03-11T00:41:23.468 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2021-03-11T00:41:23.517 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2021-03-11T00:41:23.643 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pytest (3.3.2-2) ... 2021-03-11T00:41:24.011 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python-pytest (3.3.2-2) ... 2021-03-11T00:41:24.263 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2021-03-11T00:41:24.380 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up xmlstarlet (1.6.1-2) ... 2021-03-11T00:41:24.498 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up librados2 (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:24.624 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-rsa (3.4.2-1) ... 2021-03-11T00:41:24.629 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.3) ... 2021-03-11T00:41:24.908 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libcephfs2 (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:24.955 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2021-03-11T00:41:25.036 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libcephfs-dev (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:25.160 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-rados (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:25.286 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2021-03-11T00:41:25.297 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2021-03-11T00:41:25.423 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-pytest (3.3.2-2) ... 2021-03-11T00:41:25.603 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2021-03-11T00:41:25.792 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python-pytest (3.3.2-2) ... 2021-03-11T00:41:25.905 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2021-03-11T00:41:26.043 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2021-03-11T00:41:26.161 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up xmlstarlet (1.6.1-2) ... 2021-03-11T00:41:26.231 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2021-03-11T00:41:26.261 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up librados2 (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:26.387 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-rsa (3.4.2-1) ... 2021-03-11T00:41:26.689 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up libcephfs2 (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:26.814 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up libcephfs-dev (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:26.940 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-rados (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:27.067 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2021-03-11T00:41:27.177 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-sklearn (0.19.1-3) ... 2021-03-11T00:41:27.368 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2021-03-11T00:41:27.652 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2021-03-11T00:41:27.953 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2021-03-11T00:41:28.015 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2021-03-11T00:41:28.274 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libradosstriper1 (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:28.392 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-cephfs (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:28.701 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up librgw2 (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:28.827 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-fuse (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:28.867 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-sklearn (0.19.1-3) ... 2021-03-11T00:41:29.041 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-03-11T00:41:29.041 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-03-11T00:41:29.320 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up librbd1 (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:29.446 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pecan (1.2.1-2) ... 2021-03-11T00:41:29.536 INFO:teuthology.orchestra.run.smithi119.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2021-03-11T00:41:29.569 INFO:teuthology.orchestra.run.smithi119.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2021-03-11T00:41:29.683 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2021-03-11T00:41:29.866 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2021-03-11T00:41:29.980 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up libradosstriper1 (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:30.106 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-cephfs (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:30.415 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up librgw2 (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:30.502 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mgr-modules-core (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:30.566 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up ceph-fuse (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:30.661 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pastescript (2.0.2-2) ... 2021-03-11T00:41:30.781 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-03-11T00:41:30.781 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-03-11T00:41:30.979 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-rgw (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:31.034 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up librbd1 (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:31.105 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up rbd-fuse (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:31.135 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-pecan (1.2.1-2) ... 2021-03-11T00:41:31.208 INFO:teuthology.orchestra.run.smithi124.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2021-03-11T00:41:31.223 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-rbd (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:31.241 INFO:teuthology.orchestra.run.smithi124.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2021-03-11T00:41:31.324 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-common (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:31.547 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2021-03-11T00:41:32.166 INFO:teuthology.orchestra.run.smithi119.stdout:Adding group ceph....done 2021-03-11T00:41:32.225 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up ceph-mgr-modules-core (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:32.350 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-pastescript (2.0.2-2) ... 2021-03-11T00:41:32.560 INFO:teuthology.orchestra.run.smithi119.stdout:Adding system user ceph....done 2021-03-11T00:41:32.635 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-rgw (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:32.722 INFO:teuthology.orchestra.run.smithi119.stdout:Setting system user ceph properties....done 2021-03-11T00:41:32.744 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up rbd-fuse (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:32.845 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up python3-rbd (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:32.869 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2021-03-11T00:41:32.963 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up ceph-common (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:33.098 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2021-03-11T00:41:33.383 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up radosgw (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:33.489 INFO:teuthology.orchestra.run.smithi124.stdout:Adding group ceph....done 2021-03-11T00:41:33.633 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-03-11T00:41:33.633 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-03-11T00:41:33.850 INFO:teuthology.orchestra.run.smithi124.stdout:Adding system user ceph....done 2021-03-11T00:41:34.043 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-test (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:34.050 INFO:teuthology.orchestra.run.smithi124.stdout:Setting system user ceph properties....done 2021-03-11T00:41:34.169 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-base (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:34.198 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2021-03-11T00:41:34.408 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2021-03-11T00:41:34.503 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2021-03-11T00:41:34.673 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up radosgw (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:34.804 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mgr (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:34.926 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-03-11T00:41:34.927 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-03-11T00:41:35.028 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-03-11T00:41:35.103 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-03-11T00:41:35.276 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up ceph-test (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:35.314 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mgr-diskprediction-local (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:35.401 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up ceph-base (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:35.432 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-osd (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:35.725 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2021-03-11T00:41:35.852 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-03-11T00:41:35.852 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-03-11T00:41:36.035 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up ceph-mgr (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:36.125 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mgr-cephadm (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:36.238 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-03-11T00:41:36.239 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-03-11T00:41:36.268 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:36.411 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mgr-rook (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:36.638 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up ceph-mgr-diskprediction-local (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:36.779 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mds (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:36.797 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up ceph-osd (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:37.003 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-03-11T00:41:37.003 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-03-11T00:41:37.257 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-03-11T00:41:37.258 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-03-11T00:41:37.290 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mon (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:37.495 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-03-11T00:41:37.495 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-03-11T00:41:37.508 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up ceph-mgr-cephadm (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:37.650 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:37.774 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mgr-k8sevents (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:37.793 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up ceph-mgr-rook (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:37.874 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mgr-dashboard (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:37.911 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up ceph-mds (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:37.992 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:38.111 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-03-11T00:41:38.112 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-03-11T00:41:38.139 INFO:teuthology.orchestra.run.smithi119.stdout:Processing triggers for systemd (237-3ubuntu10.44) ... 2021-03-11T00:41:38.353 INFO:teuthology.orchestra.run.smithi119.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2021-03-11T00:41:38.396 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up ceph-mon (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:38.585 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-03-11T00:41:38.586 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-03-11T00:41:38.855 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up ceph-mgr-k8sevents (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:38.981 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up ceph-mgr-dashboard (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:39.124 INFO:teuthology.orchestra.run.smithi124.stdout:Setting up ceph (15.2.9-108-g1ff7d244-1bionic) ... 2021-03-11T00:41:39.250 INFO:teuthology.orchestra.run.smithi124.stdout:Processing triggers for systemd (237-3ubuntu10.44) ... 2021-03-11T00:41:39.443 INFO:teuthology.orchestra.run.smithi124.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2021-03-11T00:41:40.414 INFO:teuthology.orchestra.run.smithi119.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2021-03-11T00:41:40.467 INFO:teuthology.orchestra.run.smithi119.stdout:ureadahead will be reprofiled on next reboot 2021-03-11T00:41:40.524 INFO:teuthology.orchestra.run.smithi119.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.4) ... 2021-03-11T00:41:40.948 INFO:teuthology.orchestra.run.smithi119.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-03-11T00:41:40.951 DEBUG:teuthology.parallel:result is None 2021-03-11T00:41:41.503 INFO:teuthology.orchestra.run.smithi124.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2021-03-11T00:41:41.722 INFO:teuthology.orchestra.run.smithi124.stdout:ureadahead will be reprofiled on next reboot 2021-03-11T00:41:41.963 INFO:teuthology.orchestra.run.smithi124.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.4) ... 2021-03-11T00:41:42.428 INFO:teuthology.orchestra.run.smithi124.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-03-11T00:41:42.431 DEBUG:teuthology.parallel:result is None 2021-03-11T00:41:42.432 DEBUG:teuthology.orchestra.run.smithi119:> sudo lsb_release -is 2021-03-11T00:41:42.488 INFO:teuthology.orchestra.run.smithi119.stdout:Ubuntu 2021-03-11T00:41:42.494 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-03-11T00:41:42.495 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2021-03-11T00:41:42.496 INFO:teuthology.packaging:ref: None 2021-03-11T00:41:42.496 INFO:teuthology.packaging:tag: None 2021-03-11T00:41:42.497 INFO:teuthology.packaging:branch: octopus 2021-03-11T00:41:42.498 INFO:teuthology.packaging:sha1: 409dc4b843da18e60e08a06ad16195db022fb227 2021-03-11T00:41:42.499 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=octopus 2021-03-11T00:41:42.728 DEBUG:teuthology.orchestra.run.smithi119:> dpkg-query -W -f '${Version}' ceph 2021-03-11T00:41:42.753 INFO:teuthology.orchestra.run.smithi119.stdout:15.2.9-108-g1ff7d244-1bionic 2021-03-11T00:41:42.754 INFO:teuthology.packaging:The installed version of ceph is 15.2.9-108-g1ff7d244-1bionic 2021-03-11T00:41:42.754 INFO:teuthology.task.install:The correct ceph version 15.2.9-108-g1ff7d244-1bionic is installed. 2021-03-11T00:41:42.755 DEBUG:teuthology.orchestra.run.smithi124:> sudo lsb_release -is 2021-03-11T00:41:42.811 INFO:teuthology.orchestra.run.smithi124.stdout:Ubuntu 2021-03-11T00:41:42.817 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-03-11T00:41:42.819 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2021-03-11T00:41:42.819 INFO:teuthology.packaging:ref: None 2021-03-11T00:41:42.819 INFO:teuthology.packaging:tag: None 2021-03-11T00:41:42.820 INFO:teuthology.packaging:branch: octopus 2021-03-11T00:41:42.820 INFO:teuthology.packaging:sha1: 409dc4b843da18e60e08a06ad16195db022fb227 2021-03-11T00:41:42.821 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=octopus 2021-03-11T00:41:43.028 DEBUG:teuthology.orchestra.run.smithi124:> dpkg-query -W -f '${Version}' ceph 2021-03-11T00:41:43.063 INFO:teuthology.orchestra.run.smithi124.stdout:15.2.9-108-g1ff7d244-1bionic 2021-03-11T00:41:43.064 INFO:teuthology.packaging:The installed version of ceph is 15.2.9-108-g1ff7d244-1bionic 2021-03-11T00:41:43.065 INFO:teuthology.task.install:The correct ceph version 15.2.9-108-g1ff7d244-1bionic is installed. 2021-03-11T00:41:43.066 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2021-03-11T00:41:43.076 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2021-03-11T00:41:43.076 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-03-11T00:41:43.093 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2021-03-11T00:41:43.093 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-03-11T00:41:43.121 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2021-03-11T00:41:43.123 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2021-03-11T00:41:43.124 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/daemon-helper 2021-03-11T00:41:43.153 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-03-11T00:41:43.209 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2021-03-11T00:41:43.210 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/usr/bin/daemon-helper 2021-03-11T00:41:43.227 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-03-11T00:41:43.283 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2021-03-11T00:41:43.284 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2021-03-11T00:41:43.284 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/adjust-ulimits 2021-03-11T00:41:43.307 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-03-11T00:41:43.365 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2021-03-11T00:41:43.365 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/usr/bin/adjust-ulimits 2021-03-11T00:41:43.381 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-03-11T00:41:43.434 INFO:teuthology.run_tasks:Running task ceph... 2021-03-11T00:41:43.677 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2021-03-11T00:41:43.678 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 777 /var/log/ceph 2021-03-11T00:41:43.687 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 777 /var/log/ceph 2021-03-11T00:41:43.702 INFO:tasks.ceph:Disabling ceph logrotate... 2021-03-11T00:41:43.703 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/logrotate.d/ceph 2021-03-11T00:41:43.744 DEBUG:teuthology.orchestra.run.smithi124:> sudo rm -f -- /etc/logrotate.d/ceph 2021-03-11T00:41:43.761 INFO:tasks.ceph:Creating extra log directories... 2021-03-11T00:41:43.762 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-03-11T00:41:43.800 DEBUG:teuthology.orchestra.run.smithi124:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-03-11T00:41:43.872 INFO:tasks.ceph:Creating ceph cluster c1... 2021-03-11T00:41:43.873 INFO:tasks.ceph:config {'conf': {'client': {'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync log trim interval': 0, 'rgw sync obj etag verify': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug filestore': 20, 'debug journal': 20, 'debug ms': 1, 'debug osd': 25}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)'], 'cpu_profile': set(), 'cluster': 'c1', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2021-03-11T00:41:43.874 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/sage-2021-03-11_00:22:56-upgrade:octopus-x-wip-sage-testing-2021-03-10-1754-distro-basic-smithi/5954563', 'branch': 'wip-sage-testing-2021-03-10-1754', 'description': 'upgrade:octopus-x/rgw-multisite/{clusters frontend overrides realm tasks upgrade/secondary}', 'email': None, 'first_in_suite': False, 'job_id': '5954563', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'sage-2021-03-11_00:22:56-upgrade:octopus-x-wip-sage-testing-2021-03-10-1754-distro-basic-smithi', 'nuke-on-error': True, 'overrides': {'admin_socket': {'branch': 'wip-sage-testing-2021-03-10-1754'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync log trim interval': 0, 'rgw sync obj etag verify': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug filestore': 20, 'debug journal': 20, 'debug ms': 1, 'debug osd': 25}}, 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '409dc4b843da18e60e08a06ad16195db022fb227', 'wait-for-scrub': False}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'branch': 'octopus', 'sha1': '409dc4b843da18e60e08a06ad16195db022fb227'}}, 'rgw': {'compression type': 'random', 'frontend': 'beast'}, 'rgw-multisite': {'realm': {'is default': True, 'name': 'test-realm'}, 'zonegroups': [{'endpoints': ['c1.client.0'], 'is_default': True, 'is_master': True, 'name': 'test-zonegroup', 'zones': [{'endpoints': ['c1.client.0'], 'is_default': True, 'is_master': True, 'name': 'test-zone1'}, {'endpoints': ['c2.client.0'], 'is_default': True, 'name': 'test-zone2'}, {'endpoints': ['c2.client.1'], 'is_pubsub': True, 'name': 'test-zone3'}]}]}, 'rgw-multisite-tests': {'args': ['tests.py', 'tests_ps.py'], 'branch': 'octopus', 'repo': 'https://github.com/ceph/ceph.git'}, 'workunit': {'branch': 'wip-sage-testing-2021-03-10-1754', 'sha1': '409dc4b843da18e60e08a06ad16195db022fb227'}}, 'owner': 'scheduled_sage@teuthology', 'priority': 70, 'repo': 'git://git.ceph.com/ceph-ci.git', 'roles': [['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0'], ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1']], 'sha1': '409dc4b843da18e60e08a06ad16195db022fb227', 'sleep_before_teardown': 0, 'suite': 'upgrade:octopus-x', 'suite_branch': 'wip-sage-testing-2021-03-10-1754', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_409dc4b843da18e60e08a06ad16195db022fb227/qa', 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph-ci.git', 'suite_sha1': '409dc4b843da18e60e08a06ad16195db022fb227', 'targets': {'smithi119.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBkw+7z94RmG19ReztArI3vjOPfFJkjaSGO5pvROxpV9zmO+L2Yys46XUKBb++5odwXIDVu7eHgCOrPnKsDbyd7VNJxGSk0qsuK0++1L8XQGrBDTjoFesokochACASxPO+9Ck7rwQz334ECXeJFEPuroH0UzS3GDGN2inhQ5VOUJMX5dSJ5o8pZ5o+1O89zprteWvLTh5RHvROH1LN+UHQUMMc97Z8tT3eh6OB6YB6S+OtTgtOPwvKV6nbBa43abd3qIVXbgLsMgd4AKkW9Su4QFL7JmGZqUyKNTVigVHiIsaQLA4K3OTAZtndROLdkKtEOziGkPAhdW7wJ6ZtEPU9', 'smithi124.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZAjNr5vAVlml6ZWyM0iMPm0+6NnbBxASsDjHySfZnRlfgNIjsgDdGmp+xM6cJykurCcFvTrSR6jGNZQ/SVZPriuysfAM4a97RSpxADK3ZIaF6Ngt13G6ltPTCpnDHjeIV+79rQb4R3WfxjXlYQLpHqmX30QKEF1Jttoy0zsmW5RDn47fnAKDoz8GtcV2q0pLtijDntA20Mra53SqsHom6N4AHW111a5xzx1fxB1KV0NvivusA+qdStJPIuEJh7zxwOyVrhySOQd+cmbGpC1oAS8sL5+Fq8KsPLnPncdPfxxmTjWm5+wn+02475zvLA/hmXXKCJXp8QytyAIXOk0c1'}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'cluster': 'c1', 'conf': {'client': {'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync log trim interval': 0, 'rgw sync obj etag verify': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug filestore': 20, 'debug journal': 20, 'debug ms': 1, 'debug osd': 25}}, 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '409dc4b843da18e60e08a06ad16195db022fb227', 'wait-for-scrub': False}}, {'ceph': {'cluster': 'c2'}}, {'parallel': ['upgrade-task']}, {'rgw': {'c1.client.0': {'port': 8000}, 'c2.client.0': {'port': 8000}, 'c2.client.1': {'port': 8001}}}, {'rgw-multisite': None}, {'rgw-multisite-tests': {'config': {'reconfigure_delay': 60}}}], 'teuthology_branch': 'master', 'teuthology_sha1': 'b6eecfe89a4363750d696160843a096384b357a8', 'timestamp': '2021-03-11_00:22:56', 'tube': 'smithi', 'upgrade-task': [{'install.upgrade': {'c2.client.0': None, 'c2.client.1': None, 'c2.mgr.x': None, 'c2.mon.a': None, 'c2.osd.0': None, 'c2.osd.1': None, 'c2.osd.2': None}}], 'user': 'sage', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2137'} 2021-03-11T00:41:43.875 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/c1.data 2021-03-11T00:41:43.901 DEBUG:teuthology.orchestra.run.smithi124:> install -d -m0755 -- /home/ubuntu/cephtest/c1.data 2021-03-11T00:41:43.912 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m0777 -- /var/run/ceph 2021-03-11T00:41:43.948 DEBUG:teuthology.orchestra.run.smithi124:> sudo install -d -m0777 -- /var/run/ceph 2021-03-11T00:41:43.971 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2021-03-11T00:41:43.971 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/scratch_devs of=/dev/stdout 2021-03-11T00:41:44.012 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2021-03-11T00:41:44.013 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2021-03-11T00:41:44.061 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2021-03-11T00:41:44.061 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-03-11T00:41:44.061 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 6h/6d Inode: 340 Links: 1 2021-03-11T00:41:44.062 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-03-11T00:41:44.062 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2021-03-11 00:39:04.496089827 +0000 2021-03-11T00:41:44.062 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2021-03-11 00:39:04.324089830 +0000 2021-03-11T00:41:44.062 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2021-03-11 00:39:04.324089830 +0000 2021-03-11T00:41:44.063 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: - 2021-03-11T00:41:44.070 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2021-03-11T00:41:44.071 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2021-03-11T00:41:44.071 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000329224 s, 1.6 MB/s 2021-03-11T00:41:44.075 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2021-03-11T00:41:44.127 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2021-03-11T00:41:44.128 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-03-11T00:41:44.128 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 6h/6d Inode: 346 Links: 1 2021-03-11T00:41:44.129 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-03-11T00:41:44.130 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2021-03-11 00:39:04.796089820 +0000 2021-03-11T00:41:44.130 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2021-03-11 00:39:04.632089824 +0000 2021-03-11T00:41:44.131 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2021-03-11 00:39:04.632089824 +0000 2021-03-11T00:41:44.131 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: - 2021-03-11T00:41:44.136 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2021-03-11T00:41:44.136 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2021-03-11T00:41:44.137 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000381764 s, 1.3 MB/s 2021-03-11T00:41:44.144 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2021-03-11T00:41:44.197 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2021-03-11T00:41:44.198 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-03-11T00:41:44.198 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 6h/6d Inode: 352 Links: 1 2021-03-11T00:41:44.198 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-03-11T00:41:44.198 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2021-03-11 00:39:05.104089813 +0000 2021-03-11T00:41:44.199 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2021-03-11 00:39:04.932089817 +0000 2021-03-11T00:41:44.199 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2021-03-11 00:39:04.932089817 +0000 2021-03-11T00:41:44.199 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: - 2021-03-11T00:41:44.207 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2021-03-11T00:41:44.207 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2021-03-11T00:41:44.207 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000363917 s, 1.4 MB/s 2021-03-11T00:41:44.212 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2021-03-11T00:41:44.261 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2021-03-11T00:41:44.262 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-03-11T00:41:44.262 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 6h/6d Inode: 359 Links: 1 2021-03-11T00:41:44.262 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-03-11T00:41:44.263 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2021-03-11 00:39:05.420089806 +0000 2021-03-11T00:41:44.263 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2021-03-11 00:39:05.244089810 +0000 2021-03-11T00:41:44.264 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2021-03-11 00:39:05.244089810 +0000 2021-03-11T00:41:44.264 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: - 2021-03-11T00:41:44.270 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2021-03-11T00:41:44.270 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2021-03-11T00:41:44.271 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000339824 s, 1.5 MB/s 2021-03-11T00:41:44.275 INFO:tasks.ceph:osd dev map: {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'} 2021-03-11T00:41:44.275 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi119.front.sepia.ceph.com'): {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'}} 2021-03-11T00:41:44.276 INFO:tasks.ceph:Generating config... 2021-03-11T00:41:44.280 INFO:tasks.ceph:[client] debug rgw = 20 2021-03-11T00:41:44.281 INFO:tasks.ceph:[client] rgw crypt require ssl = False 2021-03-11T00:41:44.281 INFO:tasks.ceph:[client] rgw crypt s3 kms backend = testing 2021-03-11T00:41:44.281 INFO:tasks.ceph:[client] rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2021-03-11T00:41:44.282 INFO:tasks.ceph:[client] rgw curl low speed time = 300 2021-03-11T00:41:44.282 INFO:tasks.ceph:[client] rgw data log num shards = 4 2021-03-11T00:41:44.282 INFO:tasks.ceph:[client] rgw md log max shards = 4 2021-03-11T00:41:44.283 INFO:tasks.ceph:[client] rgw sync log trim interval = 0 2021-03-11T00:41:44.287 INFO:tasks.ceph:[client] rgw sync obj etag verify = True 2021-03-11T00:41:44.287 INFO:tasks.ceph:[client] setgroup = ceph 2021-03-11T00:41:44.288 INFO:tasks.ceph:[client] setuser = ceph 2021-03-11T00:41:44.288 INFO:tasks.ceph:[mgr] debug mgr = 20 2021-03-11T00:41:44.288 INFO:tasks.ceph:[mgr] debug ms = 1 2021-03-11T00:41:44.289 INFO:tasks.ceph:[mon] debug mon = 20 2021-03-11T00:41:44.289 INFO:tasks.ceph:[mon] debug ms = 1 2021-03-11T00:41:44.290 INFO:tasks.ceph:[mon] debug paxos = 20 2021-03-11T00:41:44.290 INFO:tasks.ceph:[osd] debug filestore = 20 2021-03-11T00:41:44.291 INFO:tasks.ceph:[osd] debug journal = 20 2021-03-11T00:41:44.291 INFO:tasks.ceph:[osd] debug ms = 1 2021-03-11T00:41:44.291 INFO:tasks.ceph:[osd] debug osd = 25 2021-03-11T00:41:44.292 INFO:tasks.ceph:Setting up c1.mon.a... 2021-03-11T00:41:44.293 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/c1.keyring 2021-03-11T00:41:44.349 INFO:teuthology.orchestra.run.smithi119.stdout:creating /etc/ceph/c1.keyring 2021-03-11T00:41:44.352 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/c1.keyring 2021-03-11T00:41:44.428 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0644 /etc/ceph/c1.keyring 2021-03-11T00:41:44.485 DEBUG:tasks.ceph:Ceph mon addresses: [('c1.mon.a', '172.21.15.119')] 2021-03-11T00:41:44.486 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.119'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug filestore': 20, 'debug journal': 20, 'debug ms': 1, 'debug osd': 25}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync log trim interval': 0, 'rgw sync obj etag verify': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mon.a': {}} 2021-03-11T00:41:44.487 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2021-03-11T00:41:44.487 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2021-03-11T00:41:44.532 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.119 --print /home/ubuntu/cephtest/c1.monmap 2021-03-11T00:41:44.604 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:41:44.604 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:41:44.605 INFO:teuthology.orchestra.run.smithi119.stdout:monmaptool: monmap file /home/ubuntu/cephtest/c1.monmap 2021-03-11T00:41:44.605 INFO:teuthology.orchestra.run.smithi119.stdout:monmaptool: generated fsid 36fb6b5f-89a2-48e9-aefb-e01e9482c0df 2021-03-11T00:41:44.606 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 0 2021-03-11T00:41:44.606 INFO:teuthology.orchestra.run.smithi119.stdout:fsid 36fb6b5f-89a2-48e9-aefb-e01e9482c0df 2021-03-11T00:41:44.606 INFO:teuthology.orchestra.run.smithi119.stdout:last_changed 2021-03-11T00:41:44.605849+0000 2021-03-11T00:41:44.606 INFO:teuthology.orchestra.run.smithi119.stdout:created 2021-03-11T00:41:44.605849+0000 2021-03-11T00:41:44.607 INFO:teuthology.orchestra.run.smithi119.stdout:min_mon_release 0 (unknown) 2021-03-11T00:41:44.607 INFO:teuthology.orchestra.run.smithi119.stdout:0: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.a 2021-03-11T00:41:44.607 INFO:teuthology.orchestra.run.smithi119.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/c1.monmap (1 monitors) 2021-03-11T00:41:44.608 DEBUG:teuthology.orchestra.run.smithi119:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2021-03-11T00:41:44.652 INFO:tasks.ceph:Writing /etc/ceph/c1.conf for FSID 36fb6b5f-89a2-48e9-aefb-e01e9482c0df... 2021-03-11T00:41:44.653 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/c1.conf && sudo chmod 0644 /etc/ceph/c1.conf > /dev/null 2021-03-11T00:41:44.696 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/c1.conf && sudo chmod 0644 /etc/ceph/c1.conf > /dev/null 2021-03-11T00:41:44.726 INFO:teuthology.orchestra.run.smithi119.stdout:[global] 2021-03-11T00:41:44.727 INFO:teuthology.orchestra.run.smithi119.stdout: chdir = "" 2021-03-11T00:41:44.727 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-03-11T00:41:44.728 INFO:teuthology.orchestra.run.smithi119.stdout: auth supported = cephx 2021-03-11T00:41:44.728 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:41:44.729 INFO:teuthology.orchestra.run.smithi119.stdout: filestore xattr use omap = true 2021-03-11T00:41:44.729 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:41:44.729 INFO:teuthology.orchestra.run.smithi119.stdout: mon clock drift allowed = 1.000 2021-03-11T00:41:44.730 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:41:44.730 INFO:teuthology.orchestra.run.smithi119.stdout: osd crush chooseleaf type = 0 2021-03-11T00:41:44.731 INFO:teuthology.orchestra.run.smithi119.stdout: auth debug = true 2021-03-11T00:41:44.731 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:41:44.732 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on old message = true 2021-03-11T00:41:44.732 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on bug = true 2021-03-11T00:41:44.732 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:41:44.733 INFO:teuthology.orchestra.run.smithi119.stdout: mon max pg per osd = 10000 # >= luminous 2021-03-11T00:41:44.733 INFO:teuthology.orchestra.run.smithi119.stdout: mon pg warn max object skew = 0 2021-03-11T00:41:44.733 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:41:44.733 INFO:teuthology.orchestra.run.smithi119.stdout: # disable pg_autoscaler by default for new pools 2021-03-11T00:41:44.734 INFO:teuthology.orchestra.run.smithi119.stdout: osd_pool_default_pg_autoscale_mode = off 2021-03-11T00:41:44.734 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:41:44.735 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default size = 2 2021-03-11T00:41:44.735 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:41:44.735 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow primary affinity = true 2021-03-11T00:41:44.736 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow pg remap = true 2021-03-11T00:41:44.736 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on legacy crush tunables = false 2021-03-11T00:41:44.737 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on crush straw calc version zero = false 2021-03-11T00:41:44.737 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on no sortbitwise = false 2021-03-11T00:41:44.737 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on osd down out interval zero = false 2021-03-11T00:41:44.738 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on too few osds = false 2021-03-11T00:41:44.738 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-03-11T00:41:44.739 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_no_redundancy = false 2021-03-11T00:41:44.739 INFO:teuthology.orchestra.run.smithi119.stdout: mon_allow_pool_size_one = true 2021-03-11T00:41:44.740 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:41:44.740 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-03-11T00:41:44.740 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:41:44.741 INFO:teuthology.orchestra.run.smithi119.stdout: osd default data pool replay window = 5 2021-03-11T00:41:44.741 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:41:44.742 INFO:teuthology.orchestra.run.smithi119.stdout: mon allow pool delete = true 2021-03-11T00:41:44.742 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:41:44.743 INFO:teuthology.orchestra.run.smithi119.stdout: mon cluster log file level = debug 2021-03-11T00:41:44.743 INFO:teuthology.orchestra.run.smithi119.stdout: debug asserts on shutdown = true 2021-03-11T00:41:44.743 INFO:teuthology.orchestra.run.smithi119.stdout: mon health detail to clog = false 2021-03-11T00:41:44.744 INFO:teuthology.orchestra.run.smithi119.stdout: mon host = 172.21.15.119 2021-03-11T00:41:44.744 INFO:teuthology.orchestra.run.smithi119.stdout: fsid = 36fb6b5f-89a2-48e9-aefb-e01e9482c0df 2021-03-11T00:41:44.745 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:41:44.746 INFO:teuthology.orchestra.run.smithi119.stdout:[osd] 2021-03-11T00:41:44.746 INFO:teuthology.orchestra.run.smithi119.stdout: osd journal size = 100 2021-03-11T00:41:44.747 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:41:44.747 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub load threshold = 5.0 2021-03-11T00:41:44.747 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub max interval = 600 2021-03-11T00:41:44.748 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:41:44.748 INFO:teuthology.orchestra.run.smithi119.stdout: osd recover clone overlap = true 2021-03-11T00:41:44.749 INFO:teuthology.orchestra.run.smithi119.stdout: osd recovery max chunk = 1048576 2021-03-11T00:41:44.749 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:41:44.749 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug shutdown = true 2021-03-11T00:41:44.750 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug op order = true 2021-03-11T00:41:44.750 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug verify stray on activate = true 2021-03-11T00:41:44.751 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:41:44.751 INFO:teuthology.orchestra.run.smithi119.stdout: osd open classes on start = true 2021-03-11T00:41:44.752 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug pg log writeout = true 2021-03-11T00:41:44.752 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:41:44.752 INFO:teuthology.orchestra.run.smithi119.stdout: osd deep scrub update digest min age = 30 2021-03-11T00:41:44.753 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:41:44.753 INFO:teuthology.orchestra.run.smithi119.stdout: osd map max advance = 10 2021-03-11T00:41:44.754 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:41:44.755 INFO:teuthology.orchestra.run.smithi119.stdout: journal zero on create = true 2021-03-11T00:41:44.761 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:41:44.775 INFO:teuthology.orchestra.run.smithi119.stdout: filestore ondisk finisher threads = 3 2021-03-11T00:41:44.775 INFO:teuthology.orchestra.run.smithi119.stdout: filestore apply finisher threads = 3 2021-03-11T00:41:44.775 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:41:44.775 INFO:teuthology.orchestra.run.smithi119.stdout: bdev debug aio = true 2021-03-11T00:41:44.776 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug misdirected ops = true 2021-03-11T00:41:44.776 INFO:teuthology.orchestra.run.smithi119.stdout: debug filestore = 20 2021-03-11T00:41:44.776 INFO:teuthology.orchestra.run.smithi119.stdout: debug journal = 20 2021-03-11T00:41:44.776 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2021-03-11T00:41:44.776 INFO:teuthology.orchestra.run.smithi119.stdout: debug osd = 25 2021-03-11T00:41:44.777 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:41:44.781 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr] 2021-03-11T00:41:44.784 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2021-03-11T00:41:44.784 INFO:teuthology.orchestra.run.smithi119.stdout: debug mgr = 20 2021-03-11T00:41:44.784 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2021-03-11T00:41:44.784 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2021-03-11T00:41:44.784 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2021-03-11T00:41:44.785 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2021-03-11T00:41:44.785 INFO:teuthology.orchestra.run.smithi119.stdout: mgr/telemetry/nag = false 2021-03-11T00:41:44.785 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:41:44.785 INFO:teuthology.orchestra.run.smithi119.stdout:[mon] 2021-03-11T00:41:44.786 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2021-03-11T00:41:44.786 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2021-03-11T00:41:44.786 INFO:teuthology.orchestra.run.smithi119.stdout: debug paxos = 20 2021-03-11T00:41:44.786 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2021-03-11T00:41:44.786 INFO:teuthology.orchestra.run.smithi119.stdout: mon data avail warn = 5 2021-03-11T00:41:44.790 INFO:teuthology.orchestra.run.smithi119.stdout: mon mgr mkfs grace = 240 2021-03-11T00:41:44.790 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2021-03-11T00:41:44.790 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd reporter subtree level = osd 2021-03-11T00:41:44.791 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd prime pg temp = true 2021-03-11T00:41:44.791 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2021-03-11T00:41:44.792 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:41:44.792 INFO:teuthology.orchestra.run.smithi119.stdout:[client] 2021-03-11T00:41:44.793 INFO:teuthology.orchestra.run.smithi119.stdout: rgw cache enabled = true 2021-03-11T00:41:44.793 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable ops log = true 2021-03-11T00:41:44.793 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable usage log = true 2021-03-11T00:41:44.794 INFO:teuthology.orchestra.run.smithi119.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-03-11T00:41:44.794 INFO:teuthology.orchestra.run.smithi119.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-03-11T00:41:44.794 INFO:teuthology.orchestra.run.smithi119.stdout: debug rgw = 20 2021-03-11T00:41:44.794 INFO:teuthology.orchestra.run.smithi119.stdout: rgw crypt require ssl = False 2021-03-11T00:41:44.794 INFO:teuthology.orchestra.run.smithi119.stdout: rgw crypt s3 kms backend = testing 2021-03-11T00:41:44.795 INFO:teuthology.orchestra.run.smithi119.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2021-03-11T00:41:44.795 INFO:teuthology.orchestra.run.smithi119.stdout: rgw curl low speed time = 300 2021-03-11T00:41:44.795 INFO:teuthology.orchestra.run.smithi119.stdout: rgw data log num shards = 4 2021-03-11T00:41:44.795 INFO:teuthology.orchestra.run.smithi119.stdout: rgw md log max shards = 4 2021-03-11T00:41:44.796 INFO:teuthology.orchestra.run.smithi119.stdout: rgw sync log trim interval = 0 2021-03-11T00:41:44.796 INFO:teuthology.orchestra.run.smithi119.stdout: rgw sync obj etag verify = True 2021-03-11T00:41:44.796 INFO:teuthology.orchestra.run.smithi119.stdout: setgroup = ceph 2021-03-11T00:41:44.796 INFO:teuthology.orchestra.run.smithi119.stdout: setuser = ceph 2021-03-11T00:41:44.797 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.a] 2021-03-11T00:41:44.799 INFO:teuthology.orchestra.run.smithi124.stdout:[global] 2021-03-11T00:41:44.799 INFO:teuthology.orchestra.run.smithi124.stdout: chdir = "" 2021-03-11T00:41:44.799 INFO:teuthology.orchestra.run.smithi124.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-03-11T00:41:44.799 INFO:teuthology.orchestra.run.smithi124.stdout: auth supported = cephx 2021-03-11T00:41:44.800 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:41:44.800 INFO:teuthology.orchestra.run.smithi124.stdout: filestore xattr use omap = true 2021-03-11T00:41:44.800 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:41:44.800 INFO:teuthology.orchestra.run.smithi124.stdout: mon clock drift allowed = 1.000 2021-03-11T00:41:44.801 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:41:44.801 INFO:teuthology.orchestra.run.smithi124.stdout: osd crush chooseleaf type = 0 2021-03-11T00:41:44.801 INFO:teuthology.orchestra.run.smithi124.stdout: auth debug = true 2021-03-11T00:41:44.801 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:41:44.801 INFO:teuthology.orchestra.run.smithi124.stdout: ms die on old message = true 2021-03-11T00:41:44.802 INFO:teuthology.orchestra.run.smithi124.stdout: ms die on bug = true 2021-03-11T00:41:44.802 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:41:44.803 INFO:teuthology.orchestra.run.smithi124.stdout: mon max pg per osd = 10000 # >= luminous 2021-03-11T00:41:44.803 INFO:teuthology.orchestra.run.smithi124.stdout: mon pg warn max object skew = 0 2021-03-11T00:41:44.803 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:41:44.803 INFO:teuthology.orchestra.run.smithi124.stdout: # disable pg_autoscaler by default for new pools 2021-03-11T00:41:44.804 INFO:teuthology.orchestra.run.smithi124.stdout: osd_pool_default_pg_autoscale_mode = off 2021-03-11T00:41:44.804 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:41:44.804 INFO:teuthology.orchestra.run.smithi124.stdout: osd pool default size = 2 2021-03-11T00:41:44.805 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:41:44.805 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd allow primary affinity = true 2021-03-11T00:41:44.805 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd allow pg remap = true 2021-03-11T00:41:44.805 INFO:teuthology.orchestra.run.smithi124.stdout: mon warn on legacy crush tunables = false 2021-03-11T00:41:44.805 INFO:teuthology.orchestra.run.smithi124.stdout: mon warn on crush straw calc version zero = false 2021-03-11T00:41:44.806 INFO:teuthology.orchestra.run.smithi124.stdout: mon warn on no sortbitwise = false 2021-03-11T00:41:44.806 INFO:teuthology.orchestra.run.smithi124.stdout: mon warn on osd down out interval zero = false 2021-03-11T00:41:44.806 INFO:teuthology.orchestra.run.smithi124.stdout: mon warn on too few osds = false 2021-03-11T00:41:44.806 INFO:teuthology.orchestra.run.smithi124.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-03-11T00:41:44.806 INFO:teuthology.orchestra.run.smithi124.stdout: mon_warn_on_pool_no_redundancy = false 2021-03-11T00:41:44.807 INFO:teuthology.orchestra.run.smithi124.stdout: mon_allow_pool_size_one = true 2021-03-11T00:41:44.807 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:41:44.807 INFO:teuthology.orchestra.run.smithi124.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-03-11T00:41:44.807 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:41:44.808 INFO:teuthology.orchestra.run.smithi124.stdout: osd default data pool replay window = 5 2021-03-11T00:41:44.808 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:41:44.808 INFO:teuthology.orchestra.run.smithi124.stdout: mon allow pool delete = true 2021-03-11T00:41:44.808 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:41:44.808 INFO:teuthology.orchestra.run.smithi124.stdout: mon cluster log file level = debug 2021-03-11T00:41:44.809 INFO:teuthology.orchestra.run.smithi124.stdout: debug asserts on shutdown = true 2021-03-11T00:41:44.809 INFO:teuthology.orchestra.run.smithi124.stdout: mon health detail to clog = false 2021-03-11T00:41:44.809 INFO:teuthology.orchestra.run.smithi124.stdout: mon host = 172.21.15.119 2021-03-11T00:41:44.809 INFO:teuthology.orchestra.run.smithi124.stdout: fsid = 36fb6b5f-89a2-48e9-aefb-e01e9482c0df 2021-03-11T00:41:44.809 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:41:44.818 INFO:teuthology.orchestra.run.smithi124.stdout:[osd] 2021-03-11T00:41:44.818 INFO:teuthology.orchestra.run.smithi124.stdout: osd journal size = 100 2021-03-11T00:41:44.818 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:41:44.819 INFO:teuthology.orchestra.run.smithi124.stdout: osd scrub load threshold = 5.0 2021-03-11T00:41:44.819 INFO:teuthology.orchestra.run.smithi124.stdout: osd scrub max interval = 600 2021-03-11T00:41:44.819 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:41:44.819 INFO:teuthology.orchestra.run.smithi124.stdout: osd recover clone overlap = true 2021-03-11T00:41:44.820 INFO:teuthology.orchestra.run.smithi124.stdout: osd recovery max chunk = 1048576 2021-03-11T00:41:44.820 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:41:44.820 INFO:teuthology.orchestra.run.smithi124.stdout: osd debug shutdown = true 2021-03-11T00:41:44.820 INFO:teuthology.orchestra.run.smithi124.stdout: osd debug op order = true 2021-03-11T00:41:44.821 INFO:teuthology.orchestra.run.smithi124.stdout: osd debug verify stray on activate = true 2021-03-11T00:41:44.821 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:41:44.821 INFO:teuthology.orchestra.run.smithi124.stdout: osd open classes on start = true 2021-03-11T00:41:44.821 INFO:teuthology.orchestra.run.smithi124.stdout: osd debug pg log writeout = true 2021-03-11T00:41:44.822 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:41:44.822 INFO:teuthology.orchestra.run.smithi124.stdout: osd deep scrub update digest min age = 30 2021-03-11T00:41:44.822 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:41:44.822 INFO:teuthology.orchestra.run.smithi124.stdout: osd map max advance = 10 2021-03-11T00:41:44.822 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:41:44.823 INFO:teuthology.orchestra.run.smithi124.stdout: journal zero on create = true 2021-03-11T00:41:44.823 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:41:44.824 INFO:teuthology.orchestra.run.smithi124.stdout: filestore ondisk finisher threads = 3 2021-03-11T00:41:44.824 INFO:teuthology.orchestra.run.smithi124.stdout: filestore apply finisher threads = 3 2021-03-11T00:41:44.824 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:41:44.824 INFO:teuthology.orchestra.run.smithi124.stdout: bdev debug aio = true 2021-03-11T00:41:44.825 INFO:teuthology.orchestra.run.smithi124.stdout: osd debug misdirected ops = true 2021-03-11T00:41:44.825 INFO:teuthology.orchestra.run.smithi124.stdout: debug filestore = 20 2021-03-11T00:41:44.825 INFO:teuthology.orchestra.run.smithi124.stdout: debug journal = 20 2021-03-11T00:41:44.825 INFO:teuthology.orchestra.run.smithi124.stdout: debug ms = 1 2021-03-11T00:41:44.825 INFO:teuthology.orchestra.run.smithi124.stdout: debug osd = 25 2021-03-11T00:41:44.826 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:41:44.826 INFO:teuthology.orchestra.run.smithi124.stdout:[mgr] 2021-03-11T00:41:44.826 INFO:teuthology.orchestra.run.smithi124.stdout: debug ms = 1 2021-03-11T00:41:44.826 INFO:teuthology.orchestra.run.smithi124.stdout: debug mgr = 20 2021-03-11T00:41:44.827 INFO:teuthology.orchestra.run.smithi124.stdout: debug mon = 20 2021-03-11T00:41:44.827 INFO:teuthology.orchestra.run.smithi124.stdout: debug auth = 20 2021-03-11T00:41:44.827 INFO:teuthology.orchestra.run.smithi124.stdout: mon reweight min pgs per osd = 4 2021-03-11T00:41:44.827 INFO:teuthology.orchestra.run.smithi124.stdout: mon reweight min bytes per osd = 10 2021-03-11T00:41:44.827 INFO:teuthology.orchestra.run.smithi124.stdout: mgr/telemetry/nag = false 2021-03-11T00:41:44.828 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:41:44.828 INFO:teuthology.orchestra.run.smithi124.stdout:[mon] 2021-03-11T00:41:44.828 INFO:teuthology.orchestra.run.smithi124.stdout: debug ms = 1 2021-03-11T00:41:44.828 INFO:teuthology.orchestra.run.smithi124.stdout: debug mon = 20 2021-03-11T00:41:44.828 INFO:teuthology.orchestra.run.smithi124.stdout: debug paxos = 20 2021-03-11T00:41:44.829 INFO:teuthology.orchestra.run.smithi124.stdout: debug auth = 20 2021-03-11T00:41:44.829 INFO:teuthology.orchestra.run.smithi124.stdout: mon data avail warn = 5 2021-03-11T00:41:44.829 INFO:teuthology.orchestra.run.smithi124.stdout: mon mgr mkfs grace = 240 2021-03-11T00:41:44.829 INFO:teuthology.orchestra.run.smithi124.stdout: mon reweight min pgs per osd = 4 2021-03-11T00:41:44.830 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd reporter subtree level = osd 2021-03-11T00:41:44.830 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd prime pg temp = true 2021-03-11T00:41:44.830 INFO:teuthology.orchestra.run.smithi124.stdout: mon reweight min bytes per osd = 10 2021-03-11T00:41:44.830 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:41:44.830 INFO:teuthology.orchestra.run.smithi124.stdout:[client] 2021-03-11T00:41:44.831 INFO:teuthology.orchestra.run.smithi124.stdout: rgw cache enabled = true 2021-03-11T00:41:44.831 INFO:teuthology.orchestra.run.smithi124.stdout: rgw enable ops log = true 2021-03-11T00:41:44.831 INFO:teuthology.orchestra.run.smithi124.stdout: rgw enable usage log = true 2021-03-11T00:41:44.831 INFO:teuthology.orchestra.run.smithi124.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-03-11T00:41:44.831 INFO:teuthology.orchestra.run.smithi124.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-03-11T00:41:44.832 INFO:teuthology.orchestra.run.smithi124.stdout: debug rgw = 20 2021-03-11T00:41:44.832 INFO:teuthology.orchestra.run.smithi124.stdout: rgw crypt require ssl = False 2021-03-11T00:41:44.832 INFO:teuthology.orchestra.run.smithi124.stdout: rgw crypt s3 kms backend = testing 2021-03-11T00:41:44.832 INFO:teuthology.orchestra.run.smithi124.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2021-03-11T00:41:44.833 INFO:teuthology.orchestra.run.smithi124.stdout: rgw curl low speed time = 300 2021-03-11T00:41:44.833 INFO:teuthology.orchestra.run.smithi124.stdout: rgw data log num shards = 4 2021-03-11T00:41:44.833 INFO:teuthology.orchestra.run.smithi124.stdout: rgw md log max shards = 4 2021-03-11T00:41:44.833 INFO:teuthology.orchestra.run.smithi124.stdout: rgw sync log trim interval = 0 2021-03-11T00:41:44.834 INFO:teuthology.orchestra.run.smithi124.stdout: rgw sync obj etag verify = True 2021-03-11T00:41:44.834 INFO:teuthology.orchestra.run.smithi124.stdout: setgroup = ceph 2021-03-11T00:41:44.834 INFO:teuthology.orchestra.run.smithi124.stdout: setuser = ceph 2021-03-11T00:41:44.835 INFO:teuthology.orchestra.run.smithi124.stdout:[mon.a] 2021-03-11T00:41:44.835 INFO:tasks.ceph:Creating admin key on c1.mon.a... 2021-03-11T00:41:44.836 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/c1.keyring 2021-03-11T00:41:44.887 INFO:tasks.ceph:Copying monmap to all nodes... 2021-03-11T00:41:44.888 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2021-03-11T00:41:44.888 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/c1.keyring of=/dev/stdout 2021-03-11T00:41:44.899 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2021-03-11T00:41:44.899 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/home/ubuntu/cephtest/c1.monmap of=/dev/stdout 2021-03-11T00:41:44.948 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi119.front.sepia.ceph.com 2021-03-11T00:41:44.948 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2021-03-11T00:41:44.949 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/c1.keyring 2021-03-11T00:41:44.949 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0644 /etc/ceph/c1.keyring 2021-03-11T00:41:45.014 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2021-03-11T00:41:45.015 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/c1.monmap 2021-03-11T00:41:45.064 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi124.front.sepia.ceph.com 2021-03-11T00:41:45.065 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2021-03-11T00:41:45.065 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/etc/ceph/c1.keyring 2021-03-11T00:41:45.066 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 0644 /etc/ceph/c1.keyring 2021-03-11T00:41:45.091 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2021-03-11T00:41:45.092 DEBUG:teuthology.orchestra.run.smithi124:> dd of=/home/ubuntu/cephtest/c1.monmap 2021-03-11T00:41:45.140 INFO:tasks.ceph:Setting up mon nodes... 2021-03-11T00:41:45.140 INFO:tasks.ceph:Setting up mgr nodes... 2021-03-11T00:41:45.141 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mgr/c1-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/c1-x/keyring 2021-03-11T00:41:45.184 INFO:teuthology.orchestra.run.smithi119.stdout:creating /var/lib/ceph/mgr/c1-x/keyring 2021-03-11T00:41:45.190 INFO:tasks.ceph:Setting up mds nodes... 2021-03-11T00:41:45.191 INFO:tasks.ceph_client:Setting up client nodes... 2021-03-11T00:41:45.193 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/c1.client.0.keyring && sudo chmod 0644 /etc/ceph/c1.client.0.keyring 2021-03-11T00:41:45.266 INFO:teuthology.orchestra.run.smithi119.stdout:creating /etc/ceph/c1.client.0.keyring 2021-03-11T00:41:45.278 INFO:tasks.ceph:Running mkfs on osd nodes... 2021-03-11T00:41:45.278 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi119.front.sepia.ceph.com'): {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'}} 2021-03-11T00:41:45.279 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/c1-0 2021-03-11T00:41:45.333 INFO:tasks.ceph:roles_to_devs: {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'} 2021-03-11T00:41:45.333 INFO:tasks.ceph:role: c1.osd.0 2021-03-11T00:41:45.334 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi119.front.sepia.ceph.com 2021-03-11T00:41:45.334 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2021-03-11T00:41:45.852 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2021-03-11T00:41:45.853 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2021-03-11T00:41:45.853 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-03-11T00:41:45.853 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-03-11T00:41:45.853 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2021-03-11T00:41:45.854 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-03-11T00:41:45.854 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-03-11T00:41:45.854 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-03-11T00:41:45.854 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-03-11T00:41:45.856 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2021-03-11T00:41:45.861 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/c1-0 2021-03-11T00:41:45.940 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/c1-0 2021-03-11T00:41:45.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-11T00:41:45.952 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: /sbin/restorecon: command not found 2021-03-11T00:41:45.953 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/c1-1 2021-03-11T00:41:46.009 INFO:tasks.ceph:roles_to_devs: {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'} 2021-03-11T00:41:46.009 INFO:tasks.ceph:role: c1.osd.1 2021-03-11T00:41:46.010 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi119.front.sepia.ceph.com 2021-03-11T00:41:46.010 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2021-03-11T00:41:46.530 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2021-03-11T00:41:46.531 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2021-03-11T00:41:46.531 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-03-11T00:41:46.531 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-03-11T00:41:46.532 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2021-03-11T00:41:46.532 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-03-11T00:41:46.532 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-03-11T00:41:46.532 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-03-11T00:41:46.532 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-03-11T00:41:46.534 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2021-03-11T00:41:46.538 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/c1-1 2021-03-11T00:41:46.597 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/c1-1 2021-03-11T00:41:46.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-11T00:41:46.610 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: /sbin/restorecon: command not found 2021-03-11T00:41:46.610 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/c1-2 2021-03-11T00:41:46.665 INFO:tasks.ceph:roles_to_devs: {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'} 2021-03-11T00:41:46.666 INFO:tasks.ceph:role: c1.osd.2 2021-03-11T00:41:46.667 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi119.front.sepia.ceph.com 2021-03-11T00:41:46.667 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2021-03-11T00:41:47.161 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2021-03-11T00:41:47.162 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2021-03-11T00:41:47.162 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-03-11T00:41:47.163 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-03-11T00:41:47.163 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2021-03-11T00:41:47.163 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-03-11T00:41:47.163 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-03-11T00:41:47.164 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-03-11T00:41:47.164 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-03-11T00:41:47.166 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2021-03-11T00:41:47.167 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/c1-2 2021-03-11T00:41:47.231 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/c1-2 2021-03-11T00:41:47.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-11T00:41:47.244 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: /sbin/restorecon: command not found 2021-03-11T00:41:47.244 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c1 --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/c1.monmap 2021-03-11T00:41:47.352 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:41:47.351+0000 7f88188a3d80 -1 auth: error reading file: /var/lib/ceph/osd/c1-0/keyring: can't open /var/lib/ceph/osd/c1-0/keyring: (2) No such file or directory 2021-03-11T00:41:47.353 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:41:47.351+0000 7f88188a3d80 -1 created new key in keyring /var/lib/ceph/osd/c1-0/keyring 2021-03-11T00:41:47.353 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:41:47.351+0000 7f88188a3d80 -1 bluestore(/var/lib/ceph/osd/c1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-0/block: (2) No such file or directory 2021-03-11T00:41:47.353 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:41:47.351+0000 7f88188a3d80 -1 bluestore(/var/lib/ceph/osd/c1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-0/block: (2) No such file or directory 2021-03-11T00:41:47.354 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:41:47.351+0000 7f88188a3d80 -1 bluestore(/var/lib/ceph/osd/c1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-0/block: (2) No such file or directory 2021-03-11T00:41:47.356 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:41:47.355+0000 7f88188a3d80 -1 bluestore(/var/lib/ceph/osd/c1-0) _read_fsid unparsable uuid 2021-03-11T00:41:47.378 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:41:47.375+0000 7f88188a3d80 -1 freelist read_size_meta_from_db missing size meta in DB 2021-03-11T00:41:49.277 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c1-0 2021-03-11T00:41:49.295 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c1 --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/c1.monmap 2021-03-11T00:41:49.379 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:41:49.375+0000 7f8ea6e85d80 -1 auth: error reading file: /var/lib/ceph/osd/c1-1/keyring: can't open /var/lib/ceph/osd/c1-1/keyring: (2) No such file or directory 2021-03-11T00:41:49.380 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:41:49.375+0000 7f8ea6e85d80 -1 created new key in keyring /var/lib/ceph/osd/c1-1/keyring 2021-03-11T00:41:49.380 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:41:49.375+0000 7f8ea6e85d80 -1 bluestore(/var/lib/ceph/osd/c1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-1/block: (2) No such file or directory 2021-03-11T00:41:49.380 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:41:49.375+0000 7f8ea6e85d80 -1 bluestore(/var/lib/ceph/osd/c1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-1/block: (2) No such file or directory 2021-03-11T00:41:49.381 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:41:49.375+0000 7f8ea6e85d80 -1 bluestore(/var/lib/ceph/osd/c1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-1/block: (2) No such file or directory 2021-03-11T00:41:49.381 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:41:49.375+0000 7f8ea6e85d80 -1 bluestore(/var/lib/ceph/osd/c1-1) _read_fsid unparsable uuid 2021-03-11T00:41:49.395 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:41:49.395+0000 7f8ea6e85d80 -1 freelist read_size_meta_from_db missing size meta in DB 2021-03-11T00:41:51.289 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c1-1 2021-03-11T00:41:51.346 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c1 --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/c1.monmap 2021-03-11T00:41:51.425 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:41:51.423+0000 7f44e36bbd80 -1 auth: error reading file: /var/lib/ceph/osd/c1-2/keyring: can't open /var/lib/ceph/osd/c1-2/keyring: (2) No such file or directory 2021-03-11T00:41:51.425 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:41:51.423+0000 7f44e36bbd80 -1 created new key in keyring /var/lib/ceph/osd/c1-2/keyring 2021-03-11T00:41:51.426 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:41:51.423+0000 7f44e36bbd80 -1 bluestore(/var/lib/ceph/osd/c1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-2/block: (2) No such file or directory 2021-03-11T00:41:51.426 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:41:51.423+0000 7f44e36bbd80 -1 bluestore(/var/lib/ceph/osd/c1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-2/block: (2) No such file or directory 2021-03-11T00:41:51.427 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:41:51.423+0000 7f44e36bbd80 -1 bluestore(/var/lib/ceph/osd/c1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-2/block: (2) No such file or directory 2021-03-11T00:41:51.428 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:41:51.423+0000 7f44e36bbd80 -1 bluestore(/var/lib/ceph/osd/c1-2) _read_fsid unparsable uuid 2021-03-11T00:41:51.444 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:41:51.443+0000 7f44e36bbd80 -1 freelist read_size_meta_from_db missing size meta in DB 2021-03-11T00:41:53.332 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c1-2 2021-03-11T00:41:53.390 INFO:tasks.ceph:Reading keys from all nodes... 2021-03-11T00:41:53.391 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2021-03-11T00:41:53.392 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/mgr/c1-x/keyring of=/dev/stdout 2021-03-11T00:41:53.450 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2021-03-11T00:41:53.451 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/c1-0/keyring of=/dev/stdout 2021-03-11T00:41:53.508 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2021-03-11T00:41:53.508 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/c1-1/keyring of=/dev/stdout 2021-03-11T00:41:53.566 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2021-03-11T00:41:53.567 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/c1-2/keyring of=/dev/stdout 2021-03-11T00:41:53.624 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2021-03-11T00:41:53.625 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/c1.client.0.keyring of=/dev/stdout 2021-03-11T00:41:53.677 INFO:tasks.ceph:Adding keys to all mons... 2021-03-11T00:41:53.678 DEBUG:teuthology.orchestra.run.smithi119:> sudo tee -a /etc/ceph/c1.keyring 2021-03-11T00:41:53.734 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr.x] 2021-03-11T00:41:53.734 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBJZ0lgGpYRCxAASn8JRA95RXPfOgA2asLNAA== 2021-03-11T00:41:53.734 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.0] 2021-03-11T00:41:53.735 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBLZ0lg228ZFRAAM8afkU3izdKpgmXQj8etgw== 2021-03-11T00:41:53.735 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.1] 2021-03-11T00:41:53.735 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBNZ0lgnk+HFhAAelAkiez6p0gJeHg8aGTc+g== 2021-03-11T00:41:53.735 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.2] 2021-03-11T00:41:53.735 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBPZ0lgh3JgGRAASXLqOGaYcZUCB6EfatUl9g== 2021-03-11T00:41:53.736 INFO:teuthology.orchestra.run.smithi119.stdout:[client.0] 2021-03-11T00:41:53.736 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBJZ0lgdgf2DxAARzLOPAcSDp7o18LVR6inFg== 2021-03-11T00:41:53.736 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-03-11T00:41:53.827 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-03-11T00:41:53.912 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-03-11T00:41:53.995 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-03-11T00:41:54.082 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-03-11T00:41:54.171 INFO:tasks.ceph:Running mkfs on mon nodes... 2021-03-11T00:41:54.172 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mon/c1-a 2021-03-11T00:41:54.229 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster c1 --mkfs -i a --monmap /home/ubuntu/cephtest/c1.monmap --keyring /etc/ceph/c1.keyring 2021-03-11T00:41:54.331 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mon/c1-a 2021-03-11T00:41:54.349 DEBUG:teuthology.orchestra.run.smithi119:> rm -- /home/ubuntu/cephtest/c1.monmap 2021-03-11T00:41:54.398 INFO:tasks.ceph:Starting mon daemons in cluster c1... 2021-03-11T00:41:54.399 INFO:tasks.ceph.c1.mon.a:Restarting daemon 2021-03-11T00:41:54.401 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster c1 -i a 2021-03-11T00:41:54.444 INFO:tasks.ceph.c1.mon.a:Started 2021-03-11T00:41:54.444 INFO:tasks.ceph:Starting mgr daemons in cluster c1... 2021-03-11T00:41:54.445 INFO:tasks.ceph.c1.mgr.x:Restarting daemon 2021-03-11T00:41:54.446 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster c1 -i x 2021-03-11T00:41:54.448 INFO:tasks.ceph.c1.mgr.x:Started 2021-03-11T00:41:54.449 INFO:tasks.ceph:Setting crush tunables to default 2021-03-11T00:41:54.449 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster c1 osd crush tunables default 2021-03-11T00:41:54.744 INFO:teuthology.orchestra.run.smithi119.stderr:adjusted tunables profile to default 2021-03-11T00:41:54.759 INFO:tasks.ceph:Starting osd daemons in cluster c1... 2021-03-11T00:41:54.760 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2021-03-11T00:41:54.761 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/c1-0/fsid of=/dev/stdout 2021-03-11T00:41:54.774 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2021-03-11T00:41:54.775 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/c1-1/fsid of=/dev/stdout 2021-03-11T00:41:54.833 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2021-03-11T00:41:54.834 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/c1-2/fsid of=/dev/stdout 2021-03-11T00:41:54.891 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster c1 osd new d96f80bb-3923-4677-b133-590befc9b99e 0 2021-03-11T00:41:55.170 INFO:teuthology.orchestra.run.smithi119.stdout:0 2021-03-11T00:41:55.183 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster c1 osd new 7aef951c-020e-42fa-bdd9-c3264e7333c4 1 2021-03-11T00:41:55.380 INFO:teuthology.orchestra.run.smithi119.stdout:1 2021-03-11T00:41:55.391 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster c1 osd new c1fa4a92-c65c-49a5-b2b5-dc4d35cf928c 2 2021-03-11T00:41:55.580 INFO:teuthology.orchestra.run.smithi119.stdout:2 2021-03-11T00:41:55.592 INFO:tasks.ceph.c1.osd.0:Restarting daemon 2021-03-11T00:41:55.593 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster c1 -i 0 2021-03-11T00:41:55.596 INFO:tasks.ceph.c1.osd.0:Started 2021-03-11T00:41:55.597 INFO:tasks.ceph.c1.osd.1:Restarting daemon 2021-03-11T00:41:55.597 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster c1 -i 1 2021-03-11T00:41:55.609 INFO:tasks.ceph.c1.osd.1:Started 2021-03-11T00:41:55.609 INFO:tasks.ceph.c1.osd.2:Restarting daemon 2021-03-11T00:41:55.615 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster c1 -i 2 2021-03-11T00:41:55.625 INFO:tasks.ceph.c1.osd.2:Started 2021-03-11T00:41:55.626 INFO:tasks.ceph:Waiting for OSDs to come up 2021-03-11T00:41:55.929 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c1 osd dump --format=json 2021-03-11T00:41:55.980 INFO:tasks.ceph.c1.osd.2.smithi119.stderr:2021-03-11T00:41:55.979+0000 7f37ca33bd80 -1 Falling back to public interface 2021-03-11T00:41:56.098 INFO:teuthology.misc.health.smithi119.stdout: 2021-03-11T00:41:56.098 INFO:teuthology.misc.health.smithi119.stdout:{"epoch":5,"fsid":"36fb6b5f-89a2-48e9-aefb-e01e9482c0df","created":"2021-03-11T00:41:54.569882+0000","modified":"2021-03-11T00:41:55.578930+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"d96f80bb-3923-4677-b133-590befc9b99e","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7aef951c-020e-42fa-bdd9-c3264e7333c4","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c1fa4a92-c65c-49a5-b2b5-dc4d35cf928c","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-03-11T00:41:56.110 DEBUG:teuthology.misc:0 of 3 OSDs are up 2021-03-11T00:41:56.207 INFO:tasks.ceph.c1.osd.1.smithi119.stderr:2021-03-11T00:41:56.207+0000 7f5de4e98d80 -1 Falling back to public interface 2021-03-11T00:41:56.224 INFO:tasks.ceph.c1.osd.0.smithi119.stderr:2021-03-11T00:41:56.223+0000 7f4dae1b3d80 -1 Falling back to public interface 2021-03-11T00:41:56.596 INFO:tasks.ceph.c1.osd.2.smithi119.stderr:2021-03-11T00:41:56.595+0000 7f37ca33bd80 -1 osd.2 0 log_to_monitors {default=true} 2021-03-11T00:41:56.816 INFO:tasks.ceph.c1.osd.1.smithi119.stderr:2021-03-11T00:41:56.815+0000 7f5de4e98d80 -1 osd.1 0 log_to_monitors {default=true} 2021-03-11T00:41:56.849 INFO:tasks.ceph.c1.osd.0.smithi119.stderr:2021-03-11T00:41:56.839+0000 7f4dae1b3d80 -1 osd.0 0 log_to_monitors {default=true} 2021-03-11T00:41:58.614 INFO:tasks.ceph.c1.osd.2.smithi119.stderr:2021-03-11T00:41:58.611+0000 7f37b3d87700 -1 osd.2 0 waiting for initial osdmap 2021-03-11T00:41:58.615 INFO:tasks.ceph.c1.osd.0.smithi119.stderr:2021-03-11T00:41:58.611+0000 7f4d97bff700 -1 osd.0 0 waiting for initial osdmap 2021-03-11T00:41:58.616 INFO:tasks.ceph.c1.osd.1.smithi119.stderr:2021-03-11T00:41:58.611+0000 7f5dce8e4700 -1 osd.1 0 waiting for initial osdmap 2021-03-11T00:42:02.413 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c1 osd dump --format=json 2021-03-11T00:42:02.681 INFO:teuthology.misc.health.smithi119.stdout: 2021-03-11T00:42:02.681 INFO:teuthology.misc.health.smithi119.stdout:{"epoch":10,"fsid":"36fb6b5f-89a2-48e9-aefb-e01e9482c0df","created":"2021-03-11T00:41:54.569882+0000","modified":"2021-03-11T00:42:01.625470+0000","last_up_change":"2021-03-11T00:41:59.610826+0000","last_in_change":"2021-03-11T00:41:59.610826+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-03-11T00:41:58.466471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"d96f80bb-3923-4677-b133-590befc9b99e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":13497},{"type":"v1","addr":"172.21.15.119:6817","nonce":13497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":13497},{"type":"v1","addr":"172.21.15.119:6819","nonce":13497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":13497},{"type":"v1","addr":"172.21.15.119:6823","nonce":13497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":13497},{"type":"v1","addr":"172.21.15.119:6821","nonce":13497}]},"public_addr":"172.21.15.119:6817/13497","cluster_addr":"172.21.15.119:6819/13497","heartbeat_back_addr":"172.21.15.119:6823/13497","heartbeat_front_addr":"172.21.15.119:6821/13497","state":["exists","up"]},{"osd":1,"uuid":"7aef951c-020e-42fa-bdd9-c3264e7333c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":8,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":13496},{"type":"v1","addr":"172.21.15.119:6809","nonce":13496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":13496},{"type":"v1","addr":"172.21.15.119:6811","nonce":13496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":13496},{"type":"v1","addr":"172.21.15.119:6815","nonce":13496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":13496},{"type":"v1","addr":"172.21.15.119:6813","nonce":13496}]},"public_addr":"172.21.15.119:6809/13496","cluster_addr":"172.21.15.119:6811/13496","heartbeat_back_addr":"172.21.15.119:6815/13496","heartbeat_front_addr":"172.21.15.119:6813/13496","state":["exists","up"]},{"osd":2,"uuid":"c1fa4a92-c65c-49a5-b2b5-dc4d35cf928c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":13528},{"type":"v1","addr":"172.21.15.119:6801","nonce":13528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":13528},{"type":"v1","addr":"172.21.15.119:6803","nonce":13528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":13528},{"type":"v1","addr":"172.21.15.119:6807","nonce":13528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":13528},{"type":"v1","addr":"172.21.15.119:6805","nonce":13528}]},"public_addr":"172.21.15.119:6801/13528","cluster_addr":"172.21.15.119:6803/13528","heartbeat_back_addr":"172.21.15.119:6807/13528","heartbeat_front_addr":"172.21.15.119:6805/13528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-03-11T00:41:57.799149+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-03-11T00:41:57.827227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-03-11T00:41:57.633809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-03-11T00:42:02.700 DEBUG:teuthology.misc:3 of 3 OSDs are up 2021-03-11T00:42:02.701 INFO:tasks.ceph:Creating RBD pool 2021-03-11T00:42:02.702 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster c1 osd pool create rbd 8 2021-03-11T00:42:03.655 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'rbd' created 2021-03-11T00:42:03.668 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster c1 osd pool application enable rbd rbd --yes-i-really-mean-it 2021-03-11T00:42:04.661 INFO:teuthology.orchestra.run.smithi119.stderr:enabled application 'rbd' on pool 'rbd' 2021-03-11T00:42:04.676 INFO:tasks.ceph:Starting mds daemons in cluster c1... 2021-03-11T00:42:04.677 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd dump --format=json 2021-03-11T00:42:04.677 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2021-03-11T00:42:04.948 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:04.949 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":12,"fsid":"36fb6b5f-89a2-48e9-aefb-e01e9482c0df","created":"2021-03-11T00:41:54.569882+0000","modified":"2021-03-11T00:42:04.644573+0000","last_up_change":"2021-03-11T00:41:59.610826+0000","last_in_change":"2021-03-11T00:41:59.610826+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-03-11T00:41:58.466471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-03-11T00:42:03.024873+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d96f80bb-3923-4677-b133-590befc9b99e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":13497},{"type":"v1","addr":"172.21.15.119:6817","nonce":13497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":13497},{"type":"v1","addr":"172.21.15.119:6819","nonce":13497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":13497},{"type":"v1","addr":"172.21.15.119:6823","nonce":13497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":13497},{"type":"v1","addr":"172.21.15.119:6821","nonce":13497}]},"public_addr":"172.21.15.119:6817/13497","cluster_addr":"172.21.15.119:6819/13497","heartbeat_back_addr":"172.21.15.119:6823/13497","heartbeat_front_addr":"172.21.15.119:6821/13497","state":["exists","up"]},{"osd":1,"uuid":"7aef951c-020e-42fa-bdd9-c3264e7333c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":13496},{"type":"v1","addr":"172.21.15.119:6809","nonce":13496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":13496},{"type":"v1","addr":"172.21.15.119:6811","nonce":13496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":13496},{"type":"v1","addr":"172.21.15.119:6815","nonce":13496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":13496},{"type":"v1","addr":"172.21.15.119:6813","nonce":13496}]},"public_addr":"172.21.15.119:6809/13496","cluster_addr":"172.21.15.119:6811/13496","heartbeat_back_addr":"172.21.15.119:6815/13496","heartbeat_front_addr":"172.21.15.119:6813/13496","state":["exists","up"]},{"osd":2,"uuid":"c1fa4a92-c65c-49a5-b2b5-dc4d35cf928c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":13528},{"type":"v1","addr":"172.21.15.119:6801","nonce":13528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":13528},{"type":"v1","addr":"172.21.15.119:6803","nonce":13528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":13528},{"type":"v1","addr":"172.21.15.119:6807","nonce":13528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":13528},{"type":"v1","addr":"172.21.15.119:6805","nonce":13528}]},"public_addr":"172.21.15.119:6801/13528","cluster_addr":"172.21.15.119:6803/13528","heartbeat_back_addr":"172.21.15.119:6807/13528","heartbeat_front_addr":"172.21.15.119:6805/13528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-03-11T00:41:57.799149+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-03-11T00:41:57.827227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-03-11T00:41:57.633809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-03-11T00:42:04.962 INFO:tasks.ceph.ceph_manager.c1:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-03-11T00:41:58.466471+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '10', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-03-11T00:42:03.024873+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-03-11T00:42:04.963 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd pool get device_health_metrics pg_num 2021-03-11T00:42:05.240 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2021-03-11T00:42:05.253 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd pool get rbd pg_num 2021-03-11T00:42:05.524 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 8 2021-03-11T00:42:05.538 INFO:tasks.ceph:Waiting until c1 daemons up and pgs clean... 2021-03-11T00:42:05.539 INFO:tasks.ceph.ceph_manager.c1:waiting for mgr available 2021-03-11T00:42:05.539 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 mgr dump --format=json 2021-03-11T00:42:05.847 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:05.859 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":4,"active_gid":4098,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":13310},{"type":"v1","addr":"172.21.15.119:6825","nonce":13310}]},"active_addr":"172.21.15.119:6825/13310","active_change":"2021-03-11T00:41:58.369918+0000","active_mgr_features":4540138292840890367,"available":true,"standbys":[],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading remoto library:No module named 'remoto'","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":3301900217}]},{"addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":1084967807}]},{"addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":1194033145}]}]}} 2021-03-11T00:42:05.862 INFO:tasks.ceph.ceph_manager.c1:mgr available! 2021-03-11T00:42:05.862 INFO:tasks.ceph.ceph_manager.c1:waiting for all up 2021-03-11T00:42:05.863 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd dump --format=json 2021-03-11T00:42:06.131 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:06.132 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":13,"fsid":"36fb6b5f-89a2-48e9-aefb-e01e9482c0df","created":"2021-03-11T00:41:54.569882+0000","modified":"2021-03-11T00:42:05.651432+0000","last_up_change":"2021-03-11T00:41:59.610826+0000","last_in_change":"2021-03-11T00:41:59.610826+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-03-11T00:41:58.466471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-03-11T00:42:03.024873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d96f80bb-3923-4677-b133-590befc9b99e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":13497},{"type":"v1","addr":"172.21.15.119:6817","nonce":13497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":13497},{"type":"v1","addr":"172.21.15.119:6819","nonce":13497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":13497},{"type":"v1","addr":"172.21.15.119:6823","nonce":13497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":13497},{"type":"v1","addr":"172.21.15.119:6821","nonce":13497}]},"public_addr":"172.21.15.119:6817/13497","cluster_addr":"172.21.15.119:6819/13497","heartbeat_back_addr":"172.21.15.119:6823/13497","heartbeat_front_addr":"172.21.15.119:6821/13497","state":["exists","up"]},{"osd":1,"uuid":"7aef951c-020e-42fa-bdd9-c3264e7333c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":13496},{"type":"v1","addr":"172.21.15.119:6809","nonce":13496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":13496},{"type":"v1","addr":"172.21.15.119:6811","nonce":13496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":13496},{"type":"v1","addr":"172.21.15.119:6815","nonce":13496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":13496},{"type":"v1","addr":"172.21.15.119:6813","nonce":13496}]},"public_addr":"172.21.15.119:6809/13496","cluster_addr":"172.21.15.119:6811/13496","heartbeat_back_addr":"172.21.15.119:6815/13496","heartbeat_front_addr":"172.21.15.119:6813/13496","state":["exists","up"]},{"osd":2,"uuid":"c1fa4a92-c65c-49a5-b2b5-dc4d35cf928c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":13528},{"type":"v1","addr":"172.21.15.119:6801","nonce":13528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":13528},{"type":"v1","addr":"172.21.15.119:6803","nonce":13528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":13528},{"type":"v1","addr":"172.21.15.119:6807","nonce":13528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":13528},{"type":"v1","addr":"172.21.15.119:6805","nonce":13528}]},"public_addr":"172.21.15.119:6801/13528","cluster_addr":"172.21.15.119:6803/13528","heartbeat_back_addr":"172.21.15.119:6807/13528","heartbeat_front_addr":"172.21.15.119:6805/13528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-03-11T00:41:57.799149+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-03-11T00:41:57.827227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-03-11T00:41:57.633809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-03-11T00:42:06.144 INFO:tasks.ceph.ceph_manager.c1:all up! 2021-03-11T00:42:06.145 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd dump --format=json 2021-03-11T00:42:06.417 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:06.418 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":13,"fsid":"36fb6b5f-89a2-48e9-aefb-e01e9482c0df","created":"2021-03-11T00:41:54.569882+0000","modified":"2021-03-11T00:42:05.651432+0000","last_up_change":"2021-03-11T00:41:59.610826+0000","last_in_change":"2021-03-11T00:41:59.610826+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-03-11T00:41:58.466471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-03-11T00:42:03.024873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d96f80bb-3923-4677-b133-590befc9b99e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":13497},{"type":"v1","addr":"172.21.15.119:6817","nonce":13497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":13497},{"type":"v1","addr":"172.21.15.119:6819","nonce":13497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":13497},{"type":"v1","addr":"172.21.15.119:6823","nonce":13497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":13497},{"type":"v1","addr":"172.21.15.119:6821","nonce":13497}]},"public_addr":"172.21.15.119:6817/13497","cluster_addr":"172.21.15.119:6819/13497","heartbeat_back_addr":"172.21.15.119:6823/13497","heartbeat_front_addr":"172.21.15.119:6821/13497","state":["exists","up"]},{"osd":1,"uuid":"7aef951c-020e-42fa-bdd9-c3264e7333c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":13496},{"type":"v1","addr":"172.21.15.119:6809","nonce":13496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":13496},{"type":"v1","addr":"172.21.15.119:6811","nonce":13496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":13496},{"type":"v1","addr":"172.21.15.119:6815","nonce":13496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":13496},{"type":"v1","addr":"172.21.15.119:6813","nonce":13496}]},"public_addr":"172.21.15.119:6809/13496","cluster_addr":"172.21.15.119:6811/13496","heartbeat_back_addr":"172.21.15.119:6815/13496","heartbeat_front_addr":"172.21.15.119:6813/13496","state":["exists","up"]},{"osd":2,"uuid":"c1fa4a92-c65c-49a5-b2b5-dc4d35cf928c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":13528},{"type":"v1","addr":"172.21.15.119:6801","nonce":13528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":13528},{"type":"v1","addr":"172.21.15.119:6803","nonce":13528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":13528},{"type":"v1","addr":"172.21.15.119:6807","nonce":13528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":13528},{"type":"v1","addr":"172.21.15.119:6805","nonce":13528}]},"public_addr":"172.21.15.119:6801/13528","cluster_addr":"172.21.15.119:6803/13528","heartbeat_back_addr":"172.21.15.119:6807/13528","heartbeat_front_addr":"172.21.15.119:6805/13528","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-03-11T00:41:57.799149+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-03-11T00:41:57.827227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-03-11T00:41:57.633809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-03-11T00:42:06.431 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 tell osd.0 flush_pg_stats 2021-03-11T00:42:06.582 INFO:teuthology.orchestra.run.smithi119.stdout:34359738372 2021-03-11T00:42:06.583 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 tell osd.1 flush_pg_stats 2021-03-11T00:42:06.737 INFO:teuthology.orchestra.run.smithi119.stdout:34359738371 2021-03-11T00:42:06.738 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 tell osd.2 flush_pg_stats 2021-03-11T00:42:06.903 INFO:teuthology.orchestra.run.smithi119.stdout:34359738371 2021-03-11T00:42:06.904 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd last-stat-seq osd.0 2021-03-11T00:42:07.176 INFO:teuthology.orchestra.run.smithi119.stdout:34359738371 2021-03-11T00:42:07.189 INFO:tasks.ceph.ceph_manager.c1:need seq 34359738372 got 34359738371 for osd.0 2021-03-11T00:42:08.192 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd last-stat-seq osd.0 2021-03-11T00:42:08.470 INFO:teuthology.orchestra.run.smithi119.stdout:34359738371 2021-03-11T00:42:08.490 INFO:tasks.ceph.ceph_manager.c1:need seq 34359738372 got 34359738371 for osd.0 2021-03-11T00:42:09.497 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd last-stat-seq osd.0 2021-03-11T00:42:09.773 INFO:teuthology.orchestra.run.smithi119.stdout:34359738372 2021-03-11T00:42:09.787 INFO:tasks.ceph.ceph_manager.c1:need seq 34359738372 got 34359738372 for osd.0 2021-03-11T00:42:09.787 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd last-stat-seq osd.1 2021-03-11T00:42:10.064 INFO:teuthology.orchestra.run.smithi119.stdout:34359738371 2021-03-11T00:42:10.078 INFO:tasks.ceph.ceph_manager.c1:need seq 34359738371 got 34359738371 for osd.1 2021-03-11T00:42:10.079 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd last-stat-seq osd.2 2021-03-11T00:42:10.355 INFO:teuthology.orchestra.run.smithi119.stdout:34359738371 2021-03-11T00:42:10.373 INFO:tasks.ceph.ceph_manager.c1:need seq 34359738371 got 34359738371 for osd.2 2021-03-11T00:42:10.373 INFO:tasks.ceph.ceph_manager.c1:waiting for clean 2021-03-11T00:42:10.374 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 pg dump --format=json 2021-03-11T00:42:10.640 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:10.642 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2021-03-11T00:42:10.653 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2021-03-11T00:42:10.380464+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":3146080,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":3145728,"kb_avail":311426720,"statfs":{"total":322122547200,"available":318900961280,"internally_reserved":3221225472,"allocated":335872,"data_stored":95895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3221225472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.727253"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:04.675462+0000","last_change":"2021-03-11T00:42:04.668640+0000","last_active":"2021-03-11T00:42:04.675462+0000","last_peered":"2021-03-11T00:42:04.675462+0000","last_clean":"2021-03-11T00:42:04.675462+0000","last_became_active":"2021-03-11T00:42:04.668184+0000","last_became_peered":"2021-03-11T00:42:04.668184+0000","last_unstale":"2021-03-11T00:42:04.675462+0000","last_undegraded":"2021-03-11T00:42:04.675462+0000","last_fullsized":"2021-03-11T00:42:04.675462+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_clean_scrub_stamp":"2021-03-11T00:42:03.637981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:04.675492+0000","last_change":"2021-03-11T00:42:04.668640+0000","last_active":"2021-03-11T00:42:04.675492+0000","last_peered":"2021-03-11T00:42:04.675492+0000","last_clean":"2021-03-11T00:42:04.675492+0000","last_became_active":"2021-03-11T00:42:04.668164+0000","last_became_peered":"2021-03-11T00:42:04.668164+0000","last_unstale":"2021-03-11T00:42:04.675492+0000","last_undegraded":"2021-03-11T00:42:04.675492+0000","last_fullsized":"2021-03-11T00:42:04.675492+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_clean_scrub_stamp":"2021-03-11T00:42:03.637981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:04.675425+0000","last_change":"2021-03-11T00:42:04.668206+0000","last_active":"2021-03-11T00:42:04.675425+0000","last_peered":"2021-03-11T00:42:04.675425+0000","last_clean":"2021-03-11T00:42:04.675425+0000","last_became_active":"2021-03-11T00:42:04.667932+0000","last_became_peered":"2021-03-11T00:42:04.667932+0000","last_unstale":"2021-03-11T00:42:04.675425+0000","last_undegraded":"2021-03-11T00:42:04.675425+0000","last_fullsized":"2021-03-11T00:42:04.675425+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_clean_scrub_stamp":"2021-03-11T00:42:03.637981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"9","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:04.670296+0000","last_change":"2021-03-11T00:42:04.665610+0000","last_active":"2021-03-11T00:42:04.670296+0000","last_peered":"2021-03-11T00:42:04.670296+0000","last_clean":"2021-03-11T00:42:04.670296+0000","last_became_active":"2021-03-11T00:42:04.664967+0000","last_became_peered":"2021-03-11T00:42:04.664967+0000","last_unstale":"2021-03-11T00:42:04.670296+0000","last_undegraded":"2021-03-11T00:42:04.670296+0000","last_fullsized":"2021-03-11T00:42:04.670296+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_clean_scrub_stamp":"2021-03-11T00:42:03.637981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'1","reported_seq":"13","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:04.661225+0000","last_change":"2021-03-11T00:42:00.795501+0000","last_active":"2021-03-11T00:42:04.661225+0000","last_peered":"2021-03-11T00:42:04.661225+0000","last_clean":"2021-03-11T00:42:04.661225+0000","last_became_active":"2021-03-11T00:42:00.794345+0000","last_became_peered":"2021-03-11T00:42:00.794345+0000","last_unstale":"2021-03-11T00:42:04.661225+0000","last_undegraded":"2021-03-11T00:42:04.661225+0000","last_fullsized":"2021-03-11T00:42:04.661225+0000","mapping_epoch":8,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":9,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:41:58.596565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:41:58.596565+0000","last_clean_scrub_stamp":"2021-03-11T00:41:58.596565+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:04.670268+0000","last_change":"2021-03-11T00:42:04.666090+0000","last_active":"2021-03-11T00:42:04.670268+0000","last_peered":"2021-03-11T00:42:04.670268+0000","last_clean":"2021-03-11T00:42:04.670268+0000","last_became_active":"2021-03-11T00:42:04.665276+0000","last_became_peered":"2021-03-11T00:42:04.665276+0000","last_unstale":"2021-03-11T00:42:04.670268+0000","last_undegraded":"2021-03-11T00:42:04.670268+0000","last_fullsized":"2021-03-11T00:42:04.670268+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_clean_scrub_stamp":"2021-03-11T00:42:03.637981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"9","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:04.670334+0000","last_change":"2021-03-11T00:42:04.666092+0000","last_active":"2021-03-11T00:42:04.670334+0000","last_peered":"2021-03-11T00:42:04.670334+0000","last_clean":"2021-03-11T00:42:04.670334+0000","last_became_active":"2021-03-11T00:42:04.665305+0000","last_became_peered":"2021-03-11T00:42:04.665305+0000","last_unstale":"2021-03-11T00:42:04.670334+0000","last_undegraded":"2021-03-11T00:42:04.670334+0000","last_fullsized":"2021-03-11T00:42:04.670334+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_clean_scrub_stamp":"2021-03-11T00:42:03.637981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"8","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:04.667095+0000","last_change":"2021-03-11T00:42:04.667095+0000","last_active":"2021-03-11T00:42:04.667095+0000","last_peered":"2021-03-11T00:42:04.667095+0000","last_clean":"2021-03-11T00:42:04.667095+0000","last_became_active":"2021-03-11T00:42:04.666685+0000","last_became_peered":"2021-03-11T00:42:04.666685+0000","last_unstale":"2021-03-11T00:42:04.667095+0000","last_undegraded":"2021-03-11T00:42:04.667095+0000","last_fullsized":"2021-03-11T00:42:04.667095+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_clean_scrub_stamp":"2021-03-11T00:42:03.637981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:04.675500+0000","last_change":"2021-03-11T00:42:04.668778+0000","last_active":"2021-03-11T00:42:04.675500+0000","last_peered":"2021-03-11T00:42:04.675500+0000","last_clean":"2021-03-11T00:42:04.675500+0000","last_became_active":"2021-03-11T00:42:04.668356+0000","last_became_peered":"2021-03-11T00:42:04.668356+0000","last_unstale":"2021-03-11T00:42:04.675500+0000","last_undegraded":"2021-03-11T00:42:04.675500+0000","last_fullsized":"2021-03-11T00:42:04.675500+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_clean_scrub_stamp":"2021-03-11T00:42:03.637981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738371,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":33136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048688,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808912,"statfs":{"total":107374182400,"available":106300325888,"internally_reserved":1073741824,"allocated":106496,"data_stored":29623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]},{"osd":1,"up_from":8,"seq":34359738371,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":33136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]}]}} 2021-03-11T00:42:10.655 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 pg dump --format=json 2021-03-11T00:42:10.918 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:10.920 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2021-03-11T00:42:10.931 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2021-03-11T00:42:10.380464+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":3146080,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":3145728,"kb_avail":311426720,"statfs":{"total":322122547200,"available":318900961280,"internally_reserved":3221225472,"allocated":335872,"data_stored":95895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3221225472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.727253"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:04.675462+0000","last_change":"2021-03-11T00:42:04.668640+0000","last_active":"2021-03-11T00:42:04.675462+0000","last_peered":"2021-03-11T00:42:04.675462+0000","last_clean":"2021-03-11T00:42:04.675462+0000","last_became_active":"2021-03-11T00:42:04.668184+0000","last_became_peered":"2021-03-11T00:42:04.668184+0000","last_unstale":"2021-03-11T00:42:04.675462+0000","last_undegraded":"2021-03-11T00:42:04.675462+0000","last_fullsized":"2021-03-11T00:42:04.675462+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_clean_scrub_stamp":"2021-03-11T00:42:03.637981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:04.675492+0000","last_change":"2021-03-11T00:42:04.668640+0000","last_active":"2021-03-11T00:42:04.675492+0000","last_peered":"2021-03-11T00:42:04.675492+0000","last_clean":"2021-03-11T00:42:04.675492+0000","last_became_active":"2021-03-11T00:42:04.668164+0000","last_became_peered":"2021-03-11T00:42:04.668164+0000","last_unstale":"2021-03-11T00:42:04.675492+0000","last_undegraded":"2021-03-11T00:42:04.675492+0000","last_fullsized":"2021-03-11T00:42:04.675492+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_clean_scrub_stamp":"2021-03-11T00:42:03.637981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:04.675425+0000","last_change":"2021-03-11T00:42:04.668206+0000","last_active":"2021-03-11T00:42:04.675425+0000","last_peered":"2021-03-11T00:42:04.675425+0000","last_clean":"2021-03-11T00:42:04.675425+0000","last_became_active":"2021-03-11T00:42:04.667932+0000","last_became_peered":"2021-03-11T00:42:04.667932+0000","last_unstale":"2021-03-11T00:42:04.675425+0000","last_undegraded":"2021-03-11T00:42:04.675425+0000","last_fullsized":"2021-03-11T00:42:04.675425+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_clean_scrub_stamp":"2021-03-11T00:42:03.637981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"9","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:04.670296+0000","last_change":"2021-03-11T00:42:04.665610+0000","last_active":"2021-03-11T00:42:04.670296+0000","last_peered":"2021-03-11T00:42:04.670296+0000","last_clean":"2021-03-11T00:42:04.670296+0000","last_became_active":"2021-03-11T00:42:04.664967+0000","last_became_peered":"2021-03-11T00:42:04.664967+0000","last_unstale":"2021-03-11T00:42:04.670296+0000","last_undegraded":"2021-03-11T00:42:04.670296+0000","last_fullsized":"2021-03-11T00:42:04.670296+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_clean_scrub_stamp":"2021-03-11T00:42:03.637981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'1","reported_seq":"13","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:04.661225+0000","last_change":"2021-03-11T00:42:00.795501+0000","last_active":"2021-03-11T00:42:04.661225+0000","last_peered":"2021-03-11T00:42:04.661225+0000","last_clean":"2021-03-11T00:42:04.661225+0000","last_became_active":"2021-03-11T00:42:00.794345+0000","last_became_peered":"2021-03-11T00:42:00.794345+0000","last_unstale":"2021-03-11T00:42:04.661225+0000","last_undegraded":"2021-03-11T00:42:04.661225+0000","last_fullsized":"2021-03-11T00:42:04.661225+0000","mapping_epoch":8,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":9,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:41:58.596565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:41:58.596565+0000","last_clean_scrub_stamp":"2021-03-11T00:41:58.596565+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:04.670268+0000","last_change":"2021-03-11T00:42:04.666090+0000","last_active":"2021-03-11T00:42:04.670268+0000","last_peered":"2021-03-11T00:42:04.670268+0000","last_clean":"2021-03-11T00:42:04.670268+0000","last_became_active":"2021-03-11T00:42:04.665276+0000","last_became_peered":"2021-03-11T00:42:04.665276+0000","last_unstale":"2021-03-11T00:42:04.670268+0000","last_undegraded":"2021-03-11T00:42:04.670268+0000","last_fullsized":"2021-03-11T00:42:04.670268+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_clean_scrub_stamp":"2021-03-11T00:42:03.637981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"9","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:04.670334+0000","last_change":"2021-03-11T00:42:04.666092+0000","last_active":"2021-03-11T00:42:04.670334+0000","last_peered":"2021-03-11T00:42:04.670334+0000","last_clean":"2021-03-11T00:42:04.670334+0000","last_became_active":"2021-03-11T00:42:04.665305+0000","last_became_peered":"2021-03-11T00:42:04.665305+0000","last_unstale":"2021-03-11T00:42:04.670334+0000","last_undegraded":"2021-03-11T00:42:04.670334+0000","last_fullsized":"2021-03-11T00:42:04.670334+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_clean_scrub_stamp":"2021-03-11T00:42:03.637981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"8","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:04.667095+0000","last_change":"2021-03-11T00:42:04.667095+0000","last_active":"2021-03-11T00:42:04.667095+0000","last_peered":"2021-03-11T00:42:04.667095+0000","last_clean":"2021-03-11T00:42:04.667095+0000","last_became_active":"2021-03-11T00:42:04.666685+0000","last_became_peered":"2021-03-11T00:42:04.666685+0000","last_unstale":"2021-03-11T00:42:04.667095+0000","last_undegraded":"2021-03-11T00:42:04.667095+0000","last_fullsized":"2021-03-11T00:42:04.667095+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_clean_scrub_stamp":"2021-03-11T00:42:03.637981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:04.675500+0000","last_change":"2021-03-11T00:42:04.668778+0000","last_active":"2021-03-11T00:42:04.675500+0000","last_peered":"2021-03-11T00:42:04.675500+0000","last_clean":"2021-03-11T00:42:04.675500+0000","last_became_active":"2021-03-11T00:42:04.668356+0000","last_became_peered":"2021-03-11T00:42:04.668356+0000","last_unstale":"2021-03-11T00:42:04.675500+0000","last_undegraded":"2021-03-11T00:42:04.675500+0000","last_fullsized":"2021-03-11T00:42:04.675500+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:03.637981+0000","last_clean_scrub_stamp":"2021-03-11T00:42:03.637981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738371,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":33136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048688,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808912,"statfs":{"total":107374182400,"available":106300325888,"internally_reserved":1073741824,"allocated":106496,"data_stored":29623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]},{"osd":1,"up_from":8,"seq":34359738371,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":33136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]}]}} 2021-03-11T00:42:10.932 INFO:tasks.ceph.ceph_manager.c1:clean! 2021-03-11T00:42:10.932 INFO:tasks.ceph:Waiting until ceph cluster c1 is healthy... 2021-03-11T00:42:10.933 INFO:tasks.ceph.ceph_manager.c1:wait_until_healthy 2021-03-11T00:42:10.933 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 health --format=json 2021-03-11T00:42:11.244 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:11.256 INFO:teuthology.orchestra.run.smithi119.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2021-03-11T00:42:11.257 INFO:tasks.ceph.ceph_manager.c1:wait_until_healthy done 2021-03-11T00:42:11.257 INFO:teuthology.run_tasks:Running task ceph... 2021-03-11T00:42:11.266 INFO:tasks.ceph:Creating ceph cluster c2... 2021-03-11T00:42:11.267 INFO:tasks.ceph:config {'conf': {'client': {'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync log trim interval': 0, 'rgw sync obj etag verify': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug filestore': 20, 'debug journal': 20, 'debug ms': 1, 'debug osd': 25}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)'], 'cpu_profile': set(), 'cluster': 'c2', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2021-03-11T00:42:11.268 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/sage-2021-03-11_00:22:56-upgrade:octopus-x-wip-sage-testing-2021-03-10-1754-distro-basic-smithi/5954563', 'branch': 'wip-sage-testing-2021-03-10-1754', 'description': 'upgrade:octopus-x/rgw-multisite/{clusters frontend overrides realm tasks upgrade/secondary}', 'email': None, 'first_in_suite': False, 'job_id': '5954563', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'sage-2021-03-11_00:22:56-upgrade:octopus-x-wip-sage-testing-2021-03-10-1754-distro-basic-smithi', 'nuke-on-error': True, 'overrides': {'admin_socket': {'branch': 'wip-sage-testing-2021-03-10-1754'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync log trim interval': 0, 'rgw sync obj etag verify': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug filestore': 20, 'debug journal': 20, 'debug ms': 1, 'debug osd': 25}}, 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '409dc4b843da18e60e08a06ad16195db022fb227', 'wait-for-scrub': False}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'branch': 'octopus', 'sha1': '409dc4b843da18e60e08a06ad16195db022fb227'}}, 'rgw': {'compression type': 'random', 'frontend': 'beast'}, 'rgw-multisite': {'realm': {'is default': True, 'name': 'test-realm'}, 'zonegroups': [{'endpoints': ['c1.client.0'], 'is_default': True, 'is_master': True, 'name': 'test-zonegroup', 'zones': [{'endpoints': ['c1.client.0'], 'is_default': True, 'is_master': True, 'name': 'test-zone1'}, {'endpoints': ['c2.client.0'], 'is_default': True, 'name': 'test-zone2'}, {'endpoints': ['c2.client.1'], 'is_pubsub': True, 'name': 'test-zone3'}]}]}, 'rgw-multisite-tests': {'args': ['tests.py', 'tests_ps.py'], 'branch': 'octopus', 'repo': 'https://github.com/ceph/ceph.git'}, 'workunit': {'branch': 'wip-sage-testing-2021-03-10-1754', 'sha1': '409dc4b843da18e60e08a06ad16195db022fb227'}}, 'owner': 'scheduled_sage@teuthology', 'priority': 70, 'repo': 'git://git.ceph.com/ceph-ci.git', 'roles': [['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0'], ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1']], 'sha1': '409dc4b843da18e60e08a06ad16195db022fb227', 'sleep_before_teardown': 0, 'suite': 'upgrade:octopus-x', 'suite_branch': 'wip-sage-testing-2021-03-10-1754', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_409dc4b843da18e60e08a06ad16195db022fb227/qa', 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph-ci.git', 'suite_sha1': '409dc4b843da18e60e08a06ad16195db022fb227', 'targets': {'smithi119.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBkw+7z94RmG19ReztArI3vjOPfFJkjaSGO5pvROxpV9zmO+L2Yys46XUKBb++5odwXIDVu7eHgCOrPnKsDbyd7VNJxGSk0qsuK0++1L8XQGrBDTjoFesokochACASxPO+9Ck7rwQz334ECXeJFEPuroH0UzS3GDGN2inhQ5VOUJMX5dSJ5o8pZ5o+1O89zprteWvLTh5RHvROH1LN+UHQUMMc97Z8tT3eh6OB6YB6S+OtTgtOPwvKV6nbBa43abd3qIVXbgLsMgd4AKkW9Su4QFL7JmGZqUyKNTVigVHiIsaQLA4K3OTAZtndROLdkKtEOziGkPAhdW7wJ6ZtEPU9', 'smithi124.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZAjNr5vAVlml6ZWyM0iMPm0+6NnbBxASsDjHySfZnRlfgNIjsgDdGmp+xM6cJykurCcFvTrSR6jGNZQ/SVZPriuysfAM4a97RSpxADK3ZIaF6Ngt13G6ltPTCpnDHjeIV+79rQb4R3WfxjXlYQLpHqmX30QKEF1Jttoy0zsmW5RDn47fnAKDoz8GtcV2q0pLtijDntA20Mra53SqsHom6N4AHW111a5xzx1fxB1KV0NvivusA+qdStJPIuEJh7zxwOyVrhySOQd+cmbGpC1oAS8sL5+Fq8KsPLnPncdPfxxmTjWm5+wn+02475zvLA/hmXXKCJXp8QytyAIXOk0c1'}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'cluster': 'c1', 'conf': {'client': {'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync log trim interval': 0, 'rgw sync obj etag verify': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug filestore': 20, 'debug journal': 20, 'debug ms': 1, 'debug osd': 25}}, 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '409dc4b843da18e60e08a06ad16195db022fb227', 'wait-for-scrub': False}}, {'ceph': {'cluster': 'c2', 'conf': {'client': {'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync log trim interval': 0, 'rgw sync obj etag verify': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug filestore': 20, 'debug journal': 20, 'debug ms': 1, 'debug osd': 25}}, 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '409dc4b843da18e60e08a06ad16195db022fb227', 'wait-for-scrub': False}}, {'parallel': ['upgrade-task']}, {'rgw': {'c1.client.0': {'port': 8000}, 'c2.client.0': {'port': 8000}, 'c2.client.1': {'port': 8001}}}, {'rgw-multisite': None}, {'rgw-multisite-tests': {'config': {'reconfigure_delay': 60}}}], 'teuthology_branch': 'master', 'teuthology_sha1': 'b6eecfe89a4363750d696160843a096384b357a8', 'timestamp': '2021-03-11_00:22:56', 'tube': 'smithi', 'upgrade-task': [{'install.upgrade': {'c2.client.0': None, 'c2.client.1': None, 'c2.mgr.x': None, 'c2.mon.a': None, 'c2.osd.0': None, 'c2.osd.1': None, 'c2.osd.2': None}}], 'user': 'sage', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2137'} 2021-03-11T00:42:11.269 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/c2.data 2021-03-11T00:42:11.277 DEBUG:teuthology.orchestra.run.smithi124:> install -d -m0755 -- /home/ubuntu/cephtest/c2.data 2021-03-11T00:42:11.286 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m0777 -- /var/run/ceph 2021-03-11T00:42:11.323 DEBUG:teuthology.orchestra.run.smithi124:> sudo install -d -m0777 -- /var/run/ceph 2021-03-11T00:42:11.345 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2021-03-11T00:42:11.345 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/scratch_devs of=/dev/stdout 2021-03-11T00:42:11.391 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2021-03-11T00:42:11.392 DEBUG:teuthology.orchestra.run.smithi124:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2021-03-11T00:42:11.440 INFO:teuthology.orchestra.run.smithi124.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2021-03-11T00:42:11.440 INFO:teuthology.orchestra.run.smithi124.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-03-11T00:42:11.441 INFO:teuthology.orchestra.run.smithi124.stdout:Device: 6h/6d Inode: 340 Links: 1 2021-03-11T00:42:11.441 INFO:teuthology.orchestra.run.smithi124.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-03-11T00:42:11.441 INFO:teuthology.orchestra.run.smithi124.stdout:Access: 2021-03-11 00:39:07.982712990 +0000 2021-03-11T00:42:11.441 INFO:teuthology.orchestra.run.smithi124.stdout:Modify: 2021-03-11 00:39:07.606712999 +0000 2021-03-11T00:42:11.442 INFO:teuthology.orchestra.run.smithi124.stdout:Change: 2021-03-11 00:39:07.606712999 +0000 2021-03-11T00:42:11.442 INFO:teuthology.orchestra.run.smithi124.stdout: Birth: - 2021-03-11T00:42:11.450 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records in 2021-03-11T00:42:11.450 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records out 2021-03-11T00:42:11.450 INFO:teuthology.orchestra.run.smithi124.stderr:512 bytes copied, 0.000387907 s, 1.3 MB/s 2021-03-11T00:42:11.455 DEBUG:teuthology.orchestra.run.smithi124:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2021-03-11T00:42:11.504 INFO:teuthology.orchestra.run.smithi124.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2021-03-11T00:42:11.504 INFO:teuthology.orchestra.run.smithi124.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-03-11T00:42:11.505 INFO:teuthology.orchestra.run.smithi124.stdout:Device: 6h/6d Inode: 346 Links: 1 2021-03-11T00:42:11.505 INFO:teuthology.orchestra.run.smithi124.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-03-11T00:42:11.505 INFO:teuthology.orchestra.run.smithi124.stdout:Access: 2021-03-11 00:39:08.286712984 +0000 2021-03-11T00:42:11.505 INFO:teuthology.orchestra.run.smithi124.stdout:Modify: 2021-03-11 00:39:08.138712987 +0000 2021-03-11T00:42:11.506 INFO:teuthology.orchestra.run.smithi124.stdout:Change: 2021-03-11 00:39:08.138712987 +0000 2021-03-11T00:42:11.506 INFO:teuthology.orchestra.run.smithi124.stdout: Birth: - 2021-03-11T00:42:11.515 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records in 2021-03-11T00:42:11.516 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records out 2021-03-11T00:42:11.516 INFO:teuthology.orchestra.run.smithi124.stderr:512 bytes copied, 0.000314818 s, 1.6 MB/s 2021-03-11T00:42:11.517 DEBUG:teuthology.orchestra.run.smithi124:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2021-03-11T00:42:11.568 INFO:teuthology.orchestra.run.smithi124.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2021-03-11T00:42:11.569 INFO:teuthology.orchestra.run.smithi124.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-03-11T00:42:11.569 INFO:teuthology.orchestra.run.smithi124.stdout:Device: 6h/6d Inode: 352 Links: 1 2021-03-11T00:42:11.569 INFO:teuthology.orchestra.run.smithi124.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-03-11T00:42:11.569 INFO:teuthology.orchestra.run.smithi124.stdout:Access: 2021-03-11 00:39:08.410712981 +0000 2021-03-11T00:42:11.570 INFO:teuthology.orchestra.run.smithi124.stdout:Modify: 2021-03-11 00:39:08.406712981 +0000 2021-03-11T00:42:11.570 INFO:teuthology.orchestra.run.smithi124.stdout:Change: 2021-03-11 00:39:08.406712981 +0000 2021-03-11T00:42:11.570 INFO:teuthology.orchestra.run.smithi124.stdout: Birth: - 2021-03-11T00:42:11.582 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records in 2021-03-11T00:42:11.582 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records out 2021-03-11T00:42:11.582 INFO:teuthology.orchestra.run.smithi124.stderr:512 bytes copied, 0.000322516 s, 1.6 MB/s 2021-03-11T00:42:11.583 DEBUG:teuthology.orchestra.run.smithi124:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2021-03-11T00:42:11.636 INFO:teuthology.orchestra.run.smithi124.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2021-03-11T00:42:11.636 INFO:teuthology.orchestra.run.smithi124.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-03-11T00:42:11.637 INFO:teuthology.orchestra.run.smithi124.stdout:Device: 6h/6d Inode: 358 Links: 1 2021-03-11T00:42:11.637 INFO:teuthology.orchestra.run.smithi124.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-03-11T00:42:11.637 INFO:teuthology.orchestra.run.smithi124.stdout:Access: 2021-03-11 00:39:08.902712970 +0000 2021-03-11T00:42:11.637 INFO:teuthology.orchestra.run.smithi124.stdout:Modify: 2021-03-11 00:39:08.750712973 +0000 2021-03-11T00:42:11.638 INFO:teuthology.orchestra.run.smithi124.stdout:Change: 2021-03-11 00:39:08.750712973 +0000 2021-03-11T00:42:11.638 INFO:teuthology.orchestra.run.smithi124.stdout: Birth: - 2021-03-11T00:42:11.645 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records in 2021-03-11T00:42:11.646 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records out 2021-03-11T00:42:11.646 INFO:teuthology.orchestra.run.smithi124.stderr:512 bytes copied, 0.00037295 s, 1.4 MB/s 2021-03-11T00:42:11.650 INFO:tasks.ceph:osd dev map: {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'} 2021-03-11T00:42:11.651 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi124.front.sepia.ceph.com'): {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'}} 2021-03-11T00:42:11.651 INFO:tasks.ceph:Generating config... 2021-03-11T00:42:11.653 INFO:tasks.ceph:[client] debug rgw = 20 2021-03-11T00:42:11.653 INFO:tasks.ceph:[client] rgw crypt require ssl = False 2021-03-11T00:42:11.654 INFO:tasks.ceph:[client] rgw crypt s3 kms backend = testing 2021-03-11T00:42:11.656 INFO:tasks.ceph:[client] rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2021-03-11T00:42:11.657 INFO:tasks.ceph:[client] rgw curl low speed time = 300 2021-03-11T00:42:11.657 INFO:tasks.ceph:[client] rgw data log num shards = 4 2021-03-11T00:42:11.658 INFO:tasks.ceph:[client] rgw md log max shards = 4 2021-03-11T00:42:11.658 INFO:tasks.ceph:[client] rgw sync log trim interval = 0 2021-03-11T00:42:11.658 INFO:tasks.ceph:[client] rgw sync obj etag verify = True 2021-03-11T00:42:11.659 INFO:tasks.ceph:[client] setgroup = ceph 2021-03-11T00:42:11.659 INFO:tasks.ceph:[client] setuser = ceph 2021-03-11T00:42:11.660 INFO:tasks.ceph:[mgr] debug mgr = 20 2021-03-11T00:42:11.660 INFO:tasks.ceph:[mgr] debug ms = 1 2021-03-11T00:42:11.660 INFO:tasks.ceph:[mon] debug mon = 20 2021-03-11T00:42:11.660 INFO:tasks.ceph:[mon] debug ms = 1 2021-03-11T00:42:11.661 INFO:tasks.ceph:[mon] debug paxos = 20 2021-03-11T00:42:11.661 INFO:tasks.ceph:[osd] debug filestore = 20 2021-03-11T00:42:11.661 INFO:tasks.ceph:[osd] debug journal = 20 2021-03-11T00:42:11.662 INFO:tasks.ceph:[osd] debug ms = 1 2021-03-11T00:42:11.662 INFO:tasks.ceph:[osd] debug osd = 25 2021-03-11T00:42:11.662 INFO:tasks.ceph:Setting up c2.mon.a... 2021-03-11T00:42:11.663 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/c2.keyring 2021-03-11T00:42:11.725 INFO:teuthology.orchestra.run.smithi124.stdout:creating /etc/ceph/c2.keyring 2021-03-11T00:42:11.727 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/c2.keyring 2021-03-11T00:42:11.804 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 0644 /etc/ceph/c2.keyring 2021-03-11T00:42:11.860 DEBUG:tasks.ceph:Ceph mon addresses: [('c2.mon.a', '172.21.15.124')] 2021-03-11T00:42:11.860 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.124'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug filestore': 20, 'debug journal': 20, 'debug ms': 1, 'debug osd': 25}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync log trim interval': 0, 'rgw sync obj etag verify': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mon.a': {}} 2021-03-11T00:42:11.861 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2021-03-11T00:42:11.862 DEBUG:teuthology.orchestra.run.smithi124:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2021-03-11T00:42:11.908 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.124 --print /home/ubuntu/cephtest/c2.monmap 2021-03-11T00:42:11.978 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:42:11.979 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:42:11.980 INFO:teuthology.orchestra.run.smithi124.stdout:monmaptool: monmap file /home/ubuntu/cephtest/c2.monmap 2021-03-11T00:42:11.980 INFO:teuthology.orchestra.run.smithi124.stdout:monmaptool: generated fsid 80ab3993-6eec-4177-8595-ca0a327e3d06 2021-03-11T00:42:11.980 INFO:teuthology.orchestra.run.smithi124.stdout:epoch 0 2021-03-11T00:42:11.981 INFO:teuthology.orchestra.run.smithi124.stdout:fsid 80ab3993-6eec-4177-8595-ca0a327e3d06 2021-03-11T00:42:11.981 INFO:teuthology.orchestra.run.smithi124.stdout:last_changed 2021-03-11T00:42:11.978577+0000 2021-03-11T00:42:11.981 INFO:teuthology.orchestra.run.smithi124.stdout:created 2021-03-11T00:42:11.978577+0000 2021-03-11T00:42:11.981 INFO:teuthology.orchestra.run.smithi124.stdout:min_mon_release 0 (unknown) 2021-03-11T00:42:11.981 INFO:teuthology.orchestra.run.smithi124.stdout:0: [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] mon.a 2021-03-11T00:42:11.982 INFO:teuthology.orchestra.run.smithi124.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/c2.monmap (1 monitors) 2021-03-11T00:42:11.982 DEBUG:teuthology.orchestra.run.smithi124:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2021-03-11T00:42:12.027 INFO:tasks.ceph:Writing /etc/ceph/c2.conf for FSID 80ab3993-6eec-4177-8595-ca0a327e3d06... 2021-03-11T00:42:12.028 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/c2.conf && sudo chmod 0644 /etc/ceph/c2.conf > /dev/null 2021-03-11T00:42:12.031 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/c2.conf && sudo chmod 0644 /etc/ceph/c2.conf > /dev/null 2021-03-11T00:42:12.072 INFO:teuthology.orchestra.run.smithi119.stdout:[global] 2021-03-11T00:42:12.073 INFO:teuthology.orchestra.run.smithi119.stdout: chdir = "" 2021-03-11T00:42:12.073 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-03-11T00:42:12.073 INFO:teuthology.orchestra.run.smithi119.stdout: auth supported = cephx 2021-03-11T00:42:12.073 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:12.074 INFO:teuthology.orchestra.run.smithi119.stdout: filestore xattr use omap = true 2021-03-11T00:42:12.074 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:12.074 INFO:teuthology.orchestra.run.smithi119.stdout: mon clock drift allowed = 1.000 2021-03-11T00:42:12.074 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:12.075 INFO:teuthology.orchestra.run.smithi119.stdout: osd crush chooseleaf type = 0 2021-03-11T00:42:12.075 INFO:teuthology.orchestra.run.smithi119.stdout: auth debug = true 2021-03-11T00:42:12.075 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:12.075 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on old message = true 2021-03-11T00:42:12.079 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on bug = true 2021-03-11T00:42:12.079 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:12.079 INFO:teuthology.orchestra.run.smithi119.stdout: mon max pg per osd = 10000 # >= luminous 2021-03-11T00:42:12.079 INFO:teuthology.orchestra.run.smithi119.stdout: mon pg warn max object skew = 0 2021-03-11T00:42:12.080 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:12.080 INFO:teuthology.orchestra.run.smithi119.stdout: # disable pg_autoscaler by default for new pools 2021-03-11T00:42:12.080 INFO:teuthology.orchestra.run.smithi119.stdout: osd_pool_default_pg_autoscale_mode = off 2021-03-11T00:42:12.080 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:12.081 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default size = 2 2021-03-11T00:42:12.081 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:12.081 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow primary affinity = true 2021-03-11T00:42:12.081 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow pg remap = true 2021-03-11T00:42:12.082 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on legacy crush tunables = false 2021-03-11T00:42:12.082 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on crush straw calc version zero = false 2021-03-11T00:42:12.082 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on no sortbitwise = false 2021-03-11T00:42:12.082 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on osd down out interval zero = false 2021-03-11T00:42:12.082 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on too few osds = false 2021-03-11T00:42:12.083 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-03-11T00:42:12.083 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_no_redundancy = false 2021-03-11T00:42:12.083 INFO:teuthology.orchestra.run.smithi119.stdout: mon_allow_pool_size_one = true 2021-03-11T00:42:12.083 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:12.084 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-03-11T00:42:12.084 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:12.084 INFO:teuthology.orchestra.run.smithi119.stdout: osd default data pool replay window = 5 2021-03-11T00:42:12.084 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:12.085 INFO:teuthology.orchestra.run.smithi119.stdout: mon allow pool delete = true 2021-03-11T00:42:12.086 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:12.086 INFO:teuthology.orchestra.run.smithi119.stdout: mon cluster log file level = debug 2021-03-11T00:42:12.087 INFO:teuthology.orchestra.run.smithi119.stdout: debug asserts on shutdown = true 2021-03-11T00:42:12.087 INFO:teuthology.orchestra.run.smithi119.stdout: mon health detail to clog = false 2021-03-11T00:42:12.087 INFO:teuthology.orchestra.run.smithi119.stdout: mon host = 172.21.15.124 2021-03-11T00:42:12.087 INFO:teuthology.orchestra.run.smithi119.stdout: fsid = 80ab3993-6eec-4177-8595-ca0a327e3d06 2021-03-11T00:42:12.088 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:12.088 INFO:teuthology.orchestra.run.smithi119.stdout:[osd] 2021-03-11T00:42:12.088 INFO:teuthology.orchestra.run.smithi119.stdout: osd journal size = 100 2021-03-11T00:42:12.088 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:12.089 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub load threshold = 5.0 2021-03-11T00:42:12.089 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub max interval = 600 2021-03-11T00:42:12.089 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:12.089 INFO:teuthology.orchestra.run.smithi119.stdout: osd recover clone overlap = true 2021-03-11T00:42:12.090 INFO:teuthology.orchestra.run.smithi119.stdout: osd recovery max chunk = 1048576 2021-03-11T00:42:12.090 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:12.090 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug shutdown = true 2021-03-11T00:42:12.090 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug op order = true 2021-03-11T00:42:12.091 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug verify stray on activate = true 2021-03-11T00:42:12.091 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:12.091 INFO:teuthology.orchestra.run.smithi119.stdout: osd open classes on start = true 2021-03-11T00:42:12.091 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug pg log writeout = true 2021-03-11T00:42:12.091 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:12.092 INFO:teuthology.orchestra.run.smithi119.stdout: osd deep scrub update digest min age = 30 2021-03-11T00:42:12.092 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:12.092 INFO:teuthology.orchestra.run.smithi119.stdout: osd map max advance = 10 2021-03-11T00:42:12.092 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:12.092 INFO:teuthology.orchestra.run.smithi119.stdout: journal zero on create = true 2021-03-11T00:42:12.093 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:12.093 INFO:teuthology.orchestra.run.smithi119.stdout: filestore ondisk finisher threads = 3 2021-03-11T00:42:12.093 INFO:teuthology.orchestra.run.smithi119.stdout: filestore apply finisher threads = 3 2021-03-11T00:42:12.093 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:12.094 INFO:teuthology.orchestra.run.smithi119.stdout: bdev debug aio = true 2021-03-11T00:42:12.094 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug misdirected ops = true 2021-03-11T00:42:12.095 INFO:teuthology.orchestra.run.smithi119.stdout: debug filestore = 20 2021-03-11T00:42:12.096 INFO:teuthology.orchestra.run.smithi119.stdout: debug journal = 20 2021-03-11T00:42:12.097 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2021-03-11T00:42:12.097 INFO:teuthology.orchestra.run.smithi119.stdout: debug osd = 25 2021-03-11T00:42:12.097 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:12.098 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr] 2021-03-11T00:42:12.098 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2021-03-11T00:42:12.098 INFO:teuthology.orchestra.run.smithi119.stdout: debug mgr = 20 2021-03-11T00:42:12.099 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2021-03-11T00:42:12.099 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2021-03-11T00:42:12.099 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2021-03-11T00:42:12.099 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2021-03-11T00:42:12.108 INFO:teuthology.orchestra.run.smithi119.stdout: mgr/telemetry/nag = false 2021-03-11T00:42:12.108 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:12.109 INFO:teuthology.orchestra.run.smithi119.stdout:[mon] 2021-03-11T00:42:12.109 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2021-03-11T00:42:12.109 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2021-03-11T00:42:12.109 INFO:teuthology.orchestra.run.smithi119.stdout: debug paxos = 20 2021-03-11T00:42:12.110 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2021-03-11T00:42:12.110 INFO:teuthology.orchestra.run.smithi119.stdout: mon data avail warn = 5 2021-03-11T00:42:12.110 INFO:teuthology.orchestra.run.smithi119.stdout: mon mgr mkfs grace = 240 2021-03-11T00:42:12.110 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2021-03-11T00:42:12.111 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd reporter subtree level = osd 2021-03-11T00:42:12.111 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd prime pg temp = true 2021-03-11T00:42:12.111 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2021-03-11T00:42:12.111 INFO:teuthology.orchestra.run.smithi119.stdout: 2021-03-11T00:42:12.112 INFO:teuthology.orchestra.run.smithi119.stdout:[client] 2021-03-11T00:42:12.112 INFO:teuthology.orchestra.run.smithi119.stdout: rgw cache enabled = true 2021-03-11T00:42:12.112 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable ops log = true 2021-03-11T00:42:12.112 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable usage log = true 2021-03-11T00:42:12.112 INFO:teuthology.orchestra.run.smithi119.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-03-11T00:42:12.113 INFO:teuthology.orchestra.run.smithi119.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-03-11T00:42:12.113 INFO:teuthology.orchestra.run.smithi119.stdout: debug rgw = 20 2021-03-11T00:42:12.113 INFO:teuthology.orchestra.run.smithi119.stdout: rgw crypt require ssl = False 2021-03-11T00:42:12.113 INFO:teuthology.orchestra.run.smithi119.stdout: rgw crypt s3 kms backend = testing 2021-03-11T00:42:12.114 INFO:teuthology.orchestra.run.smithi119.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2021-03-11T00:42:12.114 INFO:teuthology.orchestra.run.smithi119.stdout: rgw curl low speed time = 300 2021-03-11T00:42:12.114 INFO:teuthology.orchestra.run.smithi119.stdout: rgw data log num shards = 4 2021-03-11T00:42:12.114 INFO:teuthology.orchestra.run.smithi119.stdout: rgw md log max shards = 4 2021-03-11T00:42:12.114 INFO:teuthology.orchestra.run.smithi119.stdout: rgw sync log trim interval = 0 2021-03-11T00:42:12.115 INFO:teuthology.orchestra.run.smithi119.stdout: rgw sync obj etag verify = True 2021-03-11T00:42:12.115 INFO:teuthology.orchestra.run.smithi119.stdout: setgroup = ceph 2021-03-11T00:42:12.115 INFO:teuthology.orchestra.run.smithi119.stdout: setuser = ceph 2021-03-11T00:42:12.115 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.a] 2021-03-11T00:42:12.118 INFO:teuthology.orchestra.run.smithi124.stdout:[global] 2021-03-11T00:42:12.119 INFO:teuthology.orchestra.run.smithi124.stdout: chdir = "" 2021-03-11T00:42:12.119 INFO:teuthology.orchestra.run.smithi124.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-03-11T00:42:12.119 INFO:teuthology.orchestra.run.smithi124.stdout: auth supported = cephx 2021-03-11T00:42:12.120 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:12.120 INFO:teuthology.orchestra.run.smithi124.stdout: filestore xattr use omap = true 2021-03-11T00:42:12.120 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:12.120 INFO:teuthology.orchestra.run.smithi124.stdout: mon clock drift allowed = 1.000 2021-03-11T00:42:12.121 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:12.121 INFO:teuthology.orchestra.run.smithi124.stdout: osd crush chooseleaf type = 0 2021-03-11T00:42:12.121 INFO:teuthology.orchestra.run.smithi124.stdout: auth debug = true 2021-03-11T00:42:12.122 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:12.122 INFO:teuthology.orchestra.run.smithi124.stdout: ms die on old message = true 2021-03-11T00:42:12.122 INFO:teuthology.orchestra.run.smithi124.stdout: ms die on bug = true 2021-03-11T00:42:12.122 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:12.122 INFO:teuthology.orchestra.run.smithi124.stdout: mon max pg per osd = 10000 # >= luminous 2021-03-11T00:42:12.123 INFO:teuthology.orchestra.run.smithi124.stdout: mon pg warn max object skew = 0 2021-03-11T00:42:12.123 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:12.123 INFO:teuthology.orchestra.run.smithi124.stdout: # disable pg_autoscaler by default for new pools 2021-03-11T00:42:12.124 INFO:teuthology.orchestra.run.smithi124.stdout: osd_pool_default_pg_autoscale_mode = off 2021-03-11T00:42:12.124 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:12.124 INFO:teuthology.orchestra.run.smithi124.stdout: osd pool default size = 2 2021-03-11T00:42:12.124 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:12.125 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd allow primary affinity = true 2021-03-11T00:42:12.125 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd allow pg remap = true 2021-03-11T00:42:12.125 INFO:teuthology.orchestra.run.smithi124.stdout: mon warn on legacy crush tunables = false 2021-03-11T00:42:12.126 INFO:teuthology.orchestra.run.smithi124.stdout: mon warn on crush straw calc version zero = false 2021-03-11T00:42:12.127 INFO:teuthology.orchestra.run.smithi124.stdout: mon warn on no sortbitwise = false 2021-03-11T00:42:12.127 INFO:teuthology.orchestra.run.smithi124.stdout: mon warn on osd down out interval zero = false 2021-03-11T00:42:12.128 INFO:teuthology.orchestra.run.smithi124.stdout: mon warn on too few osds = false 2021-03-11T00:42:12.129 INFO:teuthology.orchestra.run.smithi124.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-03-11T00:42:12.129 INFO:teuthology.orchestra.run.smithi124.stdout: mon_warn_on_pool_no_redundancy = false 2021-03-11T00:42:12.129 INFO:teuthology.orchestra.run.smithi124.stdout: mon_allow_pool_size_one = true 2021-03-11T00:42:12.130 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:12.130 INFO:teuthology.orchestra.run.smithi124.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-03-11T00:42:12.130 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:12.131 INFO:teuthology.orchestra.run.smithi124.stdout: osd default data pool replay window = 5 2021-03-11T00:42:12.132 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:12.132 INFO:teuthology.orchestra.run.smithi124.stdout: mon allow pool delete = true 2021-03-11T00:42:12.132 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:12.132 INFO:teuthology.orchestra.run.smithi124.stdout: mon cluster log file level = debug 2021-03-11T00:42:12.133 INFO:teuthology.orchestra.run.smithi124.stdout: debug asserts on shutdown = true 2021-03-11T00:42:12.133 INFO:teuthology.orchestra.run.smithi124.stdout: mon health detail to clog = false 2021-03-11T00:42:12.133 INFO:teuthology.orchestra.run.smithi124.stdout: mon host = 172.21.15.124 2021-03-11T00:42:12.133 INFO:teuthology.orchestra.run.smithi124.stdout: fsid = 80ab3993-6eec-4177-8595-ca0a327e3d06 2021-03-11T00:42:12.134 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:12.134 INFO:teuthology.orchestra.run.smithi124.stdout:[osd] 2021-03-11T00:42:12.134 INFO:teuthology.orchestra.run.smithi124.stdout: osd journal size = 100 2021-03-11T00:42:12.134 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:12.135 INFO:teuthology.orchestra.run.smithi124.stdout: osd scrub load threshold = 5.0 2021-03-11T00:42:12.135 INFO:teuthology.orchestra.run.smithi124.stdout: osd scrub max interval = 600 2021-03-11T00:42:12.135 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:12.135 INFO:teuthology.orchestra.run.smithi124.stdout: osd recover clone overlap = true 2021-03-11T00:42:12.136 INFO:teuthology.orchestra.run.smithi124.stdout: osd recovery max chunk = 1048576 2021-03-11T00:42:12.136 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:12.136 INFO:teuthology.orchestra.run.smithi124.stdout: osd debug shutdown = true 2021-03-11T00:42:12.136 INFO:teuthology.orchestra.run.smithi124.stdout: osd debug op order = true 2021-03-11T00:42:12.137 INFO:teuthology.orchestra.run.smithi124.stdout: osd debug verify stray on activate = true 2021-03-11T00:42:12.137 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:12.137 INFO:teuthology.orchestra.run.smithi124.stdout: osd open classes on start = true 2021-03-11T00:42:12.137 INFO:teuthology.orchestra.run.smithi124.stdout: osd debug pg log writeout = true 2021-03-11T00:42:12.138 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:12.138 INFO:teuthology.orchestra.run.smithi124.stdout: osd deep scrub update digest min age = 30 2021-03-11T00:42:12.139 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:12.141 INFO:teuthology.orchestra.run.smithi124.stdout: osd map max advance = 10 2021-03-11T00:42:12.141 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:12.142 INFO:teuthology.orchestra.run.smithi124.stdout: journal zero on create = true 2021-03-11T00:42:12.142 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:12.142 INFO:teuthology.orchestra.run.smithi124.stdout: filestore ondisk finisher threads = 3 2021-03-11T00:42:12.143 INFO:teuthology.orchestra.run.smithi124.stdout: filestore apply finisher threads = 3 2021-03-11T00:42:12.143 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:12.143 INFO:teuthology.orchestra.run.smithi124.stdout: bdev debug aio = true 2021-03-11T00:42:12.143 INFO:teuthology.orchestra.run.smithi124.stdout: osd debug misdirected ops = true 2021-03-11T00:42:12.144 INFO:teuthology.orchestra.run.smithi124.stdout: debug filestore = 20 2021-03-11T00:42:12.144 INFO:teuthology.orchestra.run.smithi124.stdout: debug journal = 20 2021-03-11T00:42:12.144 INFO:teuthology.orchestra.run.smithi124.stdout: debug ms = 1 2021-03-11T00:42:12.144 INFO:teuthology.orchestra.run.smithi124.stdout: debug osd = 25 2021-03-11T00:42:12.145 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:12.145 INFO:teuthology.orchestra.run.smithi124.stdout:[mgr] 2021-03-11T00:42:12.145 INFO:teuthology.orchestra.run.smithi124.stdout: debug ms = 1 2021-03-11T00:42:12.145 INFO:teuthology.orchestra.run.smithi124.stdout: debug mgr = 20 2021-03-11T00:42:12.146 INFO:teuthology.orchestra.run.smithi124.stdout: debug mon = 20 2021-03-11T00:42:12.147 INFO:teuthology.orchestra.run.smithi124.stdout: debug auth = 20 2021-03-11T00:42:12.147 INFO:teuthology.orchestra.run.smithi124.stdout: mon reweight min pgs per osd = 4 2021-03-11T00:42:12.148 INFO:teuthology.orchestra.run.smithi124.stdout: mon reweight min bytes per osd = 10 2021-03-11T00:42:12.148 INFO:teuthology.orchestra.run.smithi124.stdout: mgr/telemetry/nag = false 2021-03-11T00:42:12.148 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:12.149 INFO:teuthology.orchestra.run.smithi124.stdout:[mon] 2021-03-11T00:42:12.151 INFO:teuthology.orchestra.run.smithi124.stdout: debug ms = 1 2021-03-11T00:42:12.151 INFO:teuthology.orchestra.run.smithi124.stdout: debug mon = 20 2021-03-11T00:42:12.152 INFO:teuthology.orchestra.run.smithi124.stdout: debug paxos = 20 2021-03-11T00:42:12.155 INFO:teuthology.orchestra.run.smithi124.stdout: debug auth = 20 2021-03-11T00:42:12.155 INFO:teuthology.orchestra.run.smithi124.stdout: mon data avail warn = 5 2021-03-11T00:42:12.167 INFO:teuthology.orchestra.run.smithi124.stdout: mon mgr mkfs grace = 240 2021-03-11T00:42:12.168 INFO:teuthology.orchestra.run.smithi124.stdout: mon reweight min pgs per osd = 4 2021-03-11T00:42:12.168 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd reporter subtree level = osd 2021-03-11T00:42:12.169 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd prime pg temp = true 2021-03-11T00:42:12.169 INFO:teuthology.orchestra.run.smithi124.stdout: mon reweight min bytes per osd = 10 2021-03-11T00:42:12.169 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:12.169 INFO:teuthology.orchestra.run.smithi124.stdout:[client] 2021-03-11T00:42:12.170 INFO:teuthology.orchestra.run.smithi124.stdout: rgw cache enabled = true 2021-03-11T00:42:12.170 INFO:teuthology.orchestra.run.smithi124.stdout: rgw enable ops log = true 2021-03-11T00:42:12.170 INFO:teuthology.orchestra.run.smithi124.stdout: rgw enable usage log = true 2021-03-11T00:42:12.171 INFO:teuthology.orchestra.run.smithi124.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-03-11T00:42:12.171 INFO:teuthology.orchestra.run.smithi124.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-03-11T00:42:12.171 INFO:teuthology.orchestra.run.smithi124.stdout: debug rgw = 20 2021-03-11T00:42:12.171 INFO:teuthology.orchestra.run.smithi124.stdout: rgw crypt require ssl = False 2021-03-11T00:42:12.172 INFO:teuthology.orchestra.run.smithi124.stdout: rgw crypt s3 kms backend = testing 2021-03-11T00:42:12.173 INFO:teuthology.orchestra.run.smithi124.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2021-03-11T00:42:12.173 INFO:teuthology.orchestra.run.smithi124.stdout: rgw curl low speed time = 300 2021-03-11T00:42:12.173 INFO:teuthology.orchestra.run.smithi124.stdout: rgw data log num shards = 4 2021-03-11T00:42:12.173 INFO:teuthology.orchestra.run.smithi124.stdout: rgw md log max shards = 4 2021-03-11T00:42:12.174 INFO:teuthology.orchestra.run.smithi124.stdout: rgw sync log trim interval = 0 2021-03-11T00:42:12.174 INFO:teuthology.orchestra.run.smithi124.stdout: rgw sync obj etag verify = True 2021-03-11T00:42:12.193 INFO:teuthology.orchestra.run.smithi124.stdout: setgroup = ceph 2021-03-11T00:42:12.194 INFO:teuthology.orchestra.run.smithi124.stdout: setuser = ceph 2021-03-11T00:42:12.194 INFO:teuthology.orchestra.run.smithi124.stdout:[mon.a] 2021-03-11T00:42:12.194 INFO:tasks.ceph:Creating admin key on c2.mon.a... 2021-03-11T00:42:12.195 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/c2.keyring 2021-03-11T00:42:12.242 INFO:tasks.ceph:Copying monmap to all nodes... 2021-03-11T00:42:12.243 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2021-03-11T00:42:12.243 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/etc/ceph/c2.keyring of=/dev/stdout 2021-03-11T00:42:12.249 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2021-03-11T00:42:12.249 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/home/ubuntu/cephtest/c2.monmap of=/dev/stdout 2021-03-11T00:42:12.298 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi119.front.sepia.ceph.com 2021-03-11T00:42:12.298 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2021-03-11T00:42:12.299 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/c2.keyring 2021-03-11T00:42:12.299 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0644 /etc/ceph/c2.keyring 2021-03-11T00:42:12.364 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2021-03-11T00:42:12.364 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/c2.monmap 2021-03-11T00:42:12.412 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi124.front.sepia.ceph.com 2021-03-11T00:42:12.413 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2021-03-11T00:42:12.413 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/etc/ceph/c2.keyring 2021-03-11T00:42:12.413 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 0644 /etc/ceph/c2.keyring 2021-03-11T00:42:12.438 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2021-03-11T00:42:12.439 DEBUG:teuthology.orchestra.run.smithi124:> dd of=/home/ubuntu/cephtest/c2.monmap 2021-03-11T00:42:12.488 INFO:tasks.ceph:Setting up mon nodes... 2021-03-11T00:42:12.488 INFO:tasks.ceph:Setting up mgr nodes... 2021-03-11T00:42:12.489 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/mgr/c2-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/c2-x/keyring 2021-03-11T00:42:12.574 INFO:teuthology.orchestra.run.smithi124.stdout:creating /var/lib/ceph/mgr/c2-x/keyring 2021-03-11T00:42:12.576 INFO:tasks.ceph:Setting up mds nodes... 2021-03-11T00:42:12.577 INFO:tasks.ceph_client:Setting up client nodes... 2021-03-11T00:42:12.578 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/c2.client.0.keyring && sudo chmod 0644 /etc/ceph/c2.client.0.keyring 2021-03-11T00:42:12.650 INFO:teuthology.orchestra.run.smithi124.stdout:creating /etc/ceph/c2.client.0.keyring 2021-03-11T00:42:12.662 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/c2.client.1.keyring && sudo chmod 0644 /etc/ceph/c2.client.1.keyring 2021-03-11T00:42:12.740 INFO:teuthology.orchestra.run.smithi124.stdout:creating /etc/ceph/c2.client.1.keyring 2021-03-11T00:42:12.751 INFO:tasks.ceph:Running mkfs on osd nodes... 2021-03-11T00:42:12.752 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi119.front.sepia.ceph.com'): {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi124.front.sepia.ceph.com'): {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'}} 2021-03-11T00:42:12.753 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/osd/c2-0 2021-03-11T00:42:12.808 INFO:tasks.ceph:roles_to_devs: {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'} 2021-03-11T00:42:12.809 INFO:tasks.ceph:role: c2.osd.0 2021-03-11T00:42:12.810 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi124.front.sepia.ceph.com 2021-03-11T00:42:12.810 DEBUG:teuthology.orchestra.run.smithi124:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2021-03-11T00:42:13.331 INFO:teuthology.orchestra.run.smithi124.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2021-03-11T00:42:13.331 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 attr=2, projid32bit=1 2021-03-11T00:42:13.332 INFO:teuthology.orchestra.run.smithi124.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-03-11T00:42:13.332 INFO:teuthology.orchestra.run.smithi124.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-03-11T00:42:13.332 INFO:teuthology.orchestra.run.smithi124.stdout: = sunit=0 swidth=0 blks 2021-03-11T00:42:13.332 INFO:teuthology.orchestra.run.smithi124.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-03-11T00:42:13.332 INFO:teuthology.orchestra.run.smithi124.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-03-11T00:42:13.333 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-03-11T00:42:13.333 INFO:teuthology.orchestra.run.smithi124.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-03-11T00:42:13.334 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi124.front.sepia.ceph.com -o noatime 2021-03-11T00:42:13.335 DEBUG:teuthology.orchestra.run.smithi124:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/c2-0 2021-03-11T00:42:13.440 DEBUG:teuthology.orchestra.run.smithi124:> sudo /sbin/restorecon /var/lib/ceph/osd/c2-0 2021-03-11T00:42:13.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-11T00:42:13.545 INFO:teuthology.orchestra.run.smithi124.stderr:sudo: /sbin/restorecon: command not found 2021-03-11T00:42:13.547 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/osd/c2-1 2021-03-11T00:42:13.648 INFO:tasks.ceph:roles_to_devs: {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'} 2021-03-11T00:42:13.650 INFO:tasks.ceph:role: c2.osd.1 2021-03-11T00:42:13.663 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi124.front.sepia.ceph.com 2021-03-11T00:42:13.664 DEBUG:teuthology.orchestra.run.smithi124:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2021-03-11T00:42:14.151 INFO:teuthology.orchestra.run.smithi124.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2021-03-11T00:42:14.152 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 attr=2, projid32bit=1 2021-03-11T00:42:14.153 INFO:teuthology.orchestra.run.smithi124.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-03-11T00:42:14.153 INFO:teuthology.orchestra.run.smithi124.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-03-11T00:42:14.155 INFO:teuthology.orchestra.run.smithi124.stdout: = sunit=0 swidth=0 blks 2021-03-11T00:42:14.155 INFO:teuthology.orchestra.run.smithi124.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-03-11T00:42:14.156 INFO:teuthology.orchestra.run.smithi124.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-03-11T00:42:14.156 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-03-11T00:42:14.156 INFO:teuthology.orchestra.run.smithi124.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-03-11T00:42:14.161 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi124.front.sepia.ceph.com -o noatime 2021-03-11T00:42:14.162 DEBUG:teuthology.orchestra.run.smithi124:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/c2-1 2021-03-11T00:42:14.211 DEBUG:teuthology.orchestra.run.smithi124:> sudo /sbin/restorecon /var/lib/ceph/osd/c2-1 2021-03-11T00:42:14.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-11T00:42:14.225 INFO:teuthology.orchestra.run.smithi124.stderr:sudo: /sbin/restorecon: command not found 2021-03-11T00:42:14.225 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/osd/c2-2 2021-03-11T00:42:14.281 INFO:tasks.ceph:roles_to_devs: {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'} 2021-03-11T00:42:14.281 INFO:tasks.ceph:role: c2.osd.2 2021-03-11T00:42:14.282 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi124.front.sepia.ceph.com 2021-03-11T00:42:14.282 DEBUG:teuthology.orchestra.run.smithi124:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2021-03-11T00:42:14.797 INFO:teuthology.orchestra.run.smithi124.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2021-03-11T00:42:14.797 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 attr=2, projid32bit=1 2021-03-11T00:42:14.797 INFO:teuthology.orchestra.run.smithi124.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-03-11T00:42:14.798 INFO:teuthology.orchestra.run.smithi124.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-03-11T00:42:14.798 INFO:teuthology.orchestra.run.smithi124.stdout: = sunit=0 swidth=0 blks 2021-03-11T00:42:14.799 INFO:teuthology.orchestra.run.smithi124.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-03-11T00:42:14.799 INFO:teuthology.orchestra.run.smithi124.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-03-11T00:42:14.799 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-03-11T00:42:14.799 INFO:teuthology.orchestra.run.smithi124.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-03-11T00:42:14.800 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi124.front.sepia.ceph.com -o noatime 2021-03-11T00:42:14.801 DEBUG:teuthology.orchestra.run.smithi124:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/c2-2 2021-03-11T00:42:14.857 DEBUG:teuthology.orchestra.run.smithi124:> sudo /sbin/restorecon /var/lib/ceph/osd/c2-2 2021-03-11T00:42:14.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-11T00:42:14.868 INFO:teuthology.orchestra.run.smithi124.stderr:sudo: /sbin/restorecon: command not found 2021-03-11T00:42:14.869 DEBUG:teuthology.orchestra.run.smithi124:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c2 --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/c2.monmap 2021-03-11T00:42:15.031 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:42:15.028+0000 7f84d7fb2d80 -1 auth: error reading file: /var/lib/ceph/osd/c2-0/keyring: can't open /var/lib/ceph/osd/c2-0/keyring: (2) No such file or directory 2021-03-11T00:42:15.033 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:42:15.028+0000 7f84d7fb2d80 -1 created new key in keyring /var/lib/ceph/osd/c2-0/keyring 2021-03-11T00:42:15.033 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:42:15.028+0000 7f84d7fb2d80 -1 bluestore(/var/lib/ceph/osd/c2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-0/block: (2) No such file or directory 2021-03-11T00:42:15.033 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:42:15.028+0000 7f84d7fb2d80 -1 bluestore(/var/lib/ceph/osd/c2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-0/block: (2) No such file or directory 2021-03-11T00:42:15.033 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:42:15.028+0000 7f84d7fb2d80 -1 bluestore(/var/lib/ceph/osd/c2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-0/block: (2) No such file or directory 2021-03-11T00:42:15.043 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:42:15.036+0000 7f84d7fb2d80 -1 bluestore(/var/lib/ceph/osd/c2-0) _read_fsid unparsable uuid 2021-03-11T00:42:15.072 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:42:15.068+0000 7f84d7fb2d80 -1 freelist read_size_meta_from_db missing size meta in DB 2021-03-11T00:42:16.973 DEBUG:teuthology.orchestra.run.smithi124:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c2-0 2021-03-11T00:42:16.992 DEBUG:teuthology.orchestra.run.smithi124:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c2 --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/c2.monmap 2021-03-11T00:42:17.071 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:42:17.068+0000 7f75b180cd80 -1 auth: error reading file: /var/lib/ceph/osd/c2-1/keyring: can't open /var/lib/ceph/osd/c2-1/keyring: (2) No such file or directory 2021-03-11T00:42:17.072 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:42:17.068+0000 7f75b180cd80 -1 created new key in keyring /var/lib/ceph/osd/c2-1/keyring 2021-03-11T00:42:17.072 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:42:17.068+0000 7f75b180cd80 -1 bluestore(/var/lib/ceph/osd/c2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-1/block: (2) No such file or directory 2021-03-11T00:42:17.072 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:42:17.068+0000 7f75b180cd80 -1 bluestore(/var/lib/ceph/osd/c2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-1/block: (2) No such file or directory 2021-03-11T00:42:17.072 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:42:17.068+0000 7f75b180cd80 -1 bluestore(/var/lib/ceph/osd/c2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-1/block: (2) No such file or directory 2021-03-11T00:42:17.073 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:42:17.068+0000 7f75b180cd80 -1 bluestore(/var/lib/ceph/osd/c2-1) _read_fsid unparsable uuid 2021-03-11T00:42:17.089 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:42:17.084+0000 7f75b180cd80 -1 freelist read_size_meta_from_db missing size meta in DB 2021-03-11T00:42:18.990 DEBUG:teuthology.orchestra.run.smithi124:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c2-1 2021-03-11T00:42:19.050 DEBUG:teuthology.orchestra.run.smithi124:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c2 --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/c2.monmap 2021-03-11T00:42:19.136 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:42:19.132+0000 7f56d6a38d80 -1 auth: error reading file: /var/lib/ceph/osd/c2-2/keyring: can't open /var/lib/ceph/osd/c2-2/keyring: (2) No such file or directory 2021-03-11T00:42:19.136 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:42:19.132+0000 7f56d6a38d80 -1 created new key in keyring /var/lib/ceph/osd/c2-2/keyring 2021-03-11T00:42:19.136 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:42:19.132+0000 7f56d6a38d80 -1 bluestore(/var/lib/ceph/osd/c2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-2/block: (2) No such file or directory 2021-03-11T00:42:19.137 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:42:19.132+0000 7f56d6a38d80 -1 bluestore(/var/lib/ceph/osd/c2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-2/block: (2) No such file or directory 2021-03-11T00:42:19.137 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:42:19.132+0000 7f56d6a38d80 -1 bluestore(/var/lib/ceph/osd/c2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-2/block: (2) No such file or directory 2021-03-11T00:42:19.137 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:42:19.132+0000 7f56d6a38d80 -1 bluestore(/var/lib/ceph/osd/c2-2) _read_fsid unparsable uuid 2021-03-11T00:42:19.151 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:42:19.148+0000 7f56d6a38d80 -1 freelist read_size_meta_from_db missing size meta in DB 2021-03-11T00:42:21.060 DEBUG:teuthology.orchestra.run.smithi124:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c2-2 2021-03-11T00:42:21.117 INFO:tasks.ceph:Reading keys from all nodes... 2021-03-11T00:42:21.117 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2021-03-11T00:42:21.118 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/mgr/c2-x/keyring of=/dev/stdout 2021-03-11T00:42:21.174 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2021-03-11T00:42:21.175 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/c2-0/keyring of=/dev/stdout 2021-03-11T00:42:21.231 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2021-03-11T00:42:21.232 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/c2-1/keyring of=/dev/stdout 2021-03-11T00:42:21.288 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2021-03-11T00:42:21.288 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/c2-2/keyring of=/dev/stdout 2021-03-11T00:42:21.344 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2021-03-11T00:42:21.344 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/etc/ceph/c2.client.0.keyring of=/dev/stdout 2021-03-11T00:42:21.392 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2021-03-11T00:42:21.392 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/etc/ceph/c2.client.1.keyring of=/dev/stdout 2021-03-11T00:42:21.440 INFO:tasks.ceph:Adding keys to all mons... 2021-03-11T00:42:21.441 DEBUG:teuthology.orchestra.run.smithi124:> sudo tee -a /etc/ceph/c2.keyring 2021-03-11T00:42:21.498 INFO:teuthology.orchestra.run.smithi124.stdout:[mgr.x] 2021-03-11T00:42:21.499 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQBkZ0lgf2szIhAAP5DZR5E3Hm11zElx+Go1iQ== 2021-03-11T00:42:21.499 INFO:teuthology.orchestra.run.smithi124.stdout:[osd.0] 2021-03-11T00:42:21.501 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQBnZ0lgjcbZARAAxyQ9AHwe/dTXPARYtYQ2rQ== 2021-03-11T00:42:21.501 INFO:teuthology.orchestra.run.smithi124.stdout:[osd.1] 2021-03-11T00:42:21.502 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQBpZ0lg6Po0BBAAgGNMljPHo5vl+JJeB//Qjg== 2021-03-11T00:42:21.502 INFO:teuthology.orchestra.run.smithi124.stdout:[osd.2] 2021-03-11T00:42:21.503 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQBrZ0lgr5oHCBAAlmv/xOrU7kS2UYiFpM+VhA== 2021-03-11T00:42:21.503 INFO:teuthology.orchestra.run.smithi124.stdout:[client.0] 2021-03-11T00:42:21.503 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQBkZ0lg3fm4JhAAooxsBAUs99RiOVCcOhVVgw== 2021-03-11T00:42:21.504 INFO:teuthology.orchestra.run.smithi124.stdout:[client.1] 2021-03-11T00:42:21.504 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQBkZ0lgF4wXLBAA7vFFweNnZN/8N6p7xQV2Ng== 2021-03-11T00:42:21.505 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-03-11T00:42:21.583 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-03-11T00:42:21.668 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-03-11T00:42:21.755 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-03-11T00:42:21.842 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-03-11T00:42:21.931 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-03-11T00:42:22.019 INFO:tasks.ceph:Running mkfs on mon nodes... 2021-03-11T00:42:22.019 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/mon/c2-a 2021-03-11T00:42:22.076 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster c2 --mkfs -i a --monmap /home/ubuntu/cephtest/c2.monmap --keyring /etc/ceph/c2.keyring 2021-03-11T00:42:22.177 DEBUG:teuthology.orchestra.run.smithi124:> sudo chown -R ceph:ceph /var/lib/ceph/mon/c2-a 2021-03-11T00:42:22.191 DEBUG:teuthology.orchestra.run.smithi124:> rm -- /home/ubuntu/cephtest/c2.monmap 2021-03-11T00:42:22.239 INFO:tasks.ceph:Starting mon daemons in cluster c2... 2021-03-11T00:42:22.240 INFO:tasks.ceph.c2.mon.a:Restarting daemon 2021-03-11T00:42:22.240 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster c2 -i a 2021-03-11T00:42:22.286 INFO:tasks.ceph.c2.mon.a:Started 2021-03-11T00:42:22.287 INFO:tasks.ceph:Starting mgr daemons in cluster c2... 2021-03-11T00:42:22.287 INFO:tasks.ceph.c2.mgr.x:Restarting daemon 2021-03-11T00:42:22.288 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster c2 -i x 2021-03-11T00:42:22.292 INFO:tasks.ceph.c2.mgr.x:Started 2021-03-11T00:42:22.293 INFO:tasks.ceph:Setting crush tunables to default 2021-03-11T00:42:22.293 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster c2 osd crush tunables default 2021-03-11T00:42:22.587 INFO:teuthology.orchestra.run.smithi124.stderr:adjusted tunables profile to default 2021-03-11T00:42:22.599 INFO:tasks.ceph:Starting osd daemons in cluster c2... 2021-03-11T00:42:22.600 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2021-03-11T00:42:22.600 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/c2-0/fsid of=/dev/stdout 2021-03-11T00:42:22.615 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2021-03-11T00:42:22.616 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/c2-1/fsid of=/dev/stdout 2021-03-11T00:42:22.671 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2021-03-11T00:42:22.672 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/c2-2/fsid of=/dev/stdout 2021-03-11T00:42:22.732 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster c2 osd new df1c2a49-7081-4a2b-97f0-0bc5fc0c16c9 0 2021-03-11T00:42:22.985 INFO:teuthology.orchestra.run.smithi124.stdout:0 2021-03-11T00:42:22.998 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster c2 osd new 1750f8ff-75de-41d9-829d-35fec51c6525 1 2021-03-11T00:42:23.202 INFO:teuthology.orchestra.run.smithi124.stdout:1 2021-03-11T00:42:23.214 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster c2 osd new 60760aca-cfcb-4f88-bcca-b5ce6b49b937 2 2021-03-11T00:42:23.413 INFO:teuthology.orchestra.run.smithi124.stdout:2 2021-03-11T00:42:23.426 INFO:tasks.ceph.c2.osd.0:Restarting daemon 2021-03-11T00:42:23.426 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster c2 -i 0 2021-03-11T00:42:23.431 INFO:tasks.ceph.c2.osd.0:Started 2021-03-11T00:42:23.431 INFO:tasks.ceph.c2.osd.1:Restarting daemon 2021-03-11T00:42:23.432 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster c2 -i 1 2021-03-11T00:42:23.435 INFO:tasks.ceph.c2.osd.1:Started 2021-03-11T00:42:23.436 INFO:tasks.ceph.c2.osd.2:Restarting daemon 2021-03-11T00:42:23.436 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster c2 -i 2 2021-03-11T00:42:23.442 INFO:tasks.ceph.c2.osd.2:Started 2021-03-11T00:42:23.443 INFO:tasks.ceph:Waiting for OSDs to come up 2021-03-11T00:42:23.746 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c2 osd dump --format=json 2021-03-11T00:42:23.929 INFO:teuthology.misc.health.smithi124.stdout: 2021-03-11T00:42:23.930 INFO:teuthology.misc.health.smithi124.stdout:{"epoch":5,"fsid":"80ab3993-6eec-4177-8595-ca0a327e3d06","created":"2021-03-11T00:42:22.411017+0000","modified":"2021-03-11T00:42:23.409799+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"df1c2a49-7081-4a2b-97f0-0bc5fc0c16c9","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1750f8ff-75de-41d9-829d-35fec51c6525","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"60760aca-cfcb-4f88-bcca-b5ce6b49b937","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-03-11T00:42:23.942 DEBUG:teuthology.misc:0 of 3 OSDs are up 2021-03-11T00:42:24.047 INFO:tasks.ceph.c2.osd.2.smithi124.stderr:2021-03-11T00:42:24.044+0000 7f0fc64f3d80 -1 Falling back to public interface 2021-03-11T00:42:24.048 INFO:tasks.ceph.c2.osd.1.smithi124.stderr:2021-03-11T00:42:24.044+0000 7f13b18f4d80 -1 Falling back to public interface 2021-03-11T00:42:24.059 INFO:tasks.ceph.c2.osd.0.smithi124.stderr:2021-03-11T00:42:24.056+0000 7f1ab2183d80 -1 Falling back to public interface 2021-03-11T00:42:24.657 INFO:tasks.ceph.c2.osd.2.smithi124.stderr:2021-03-11T00:42:24.648+0000 7f0fc64f3d80 -1 osd.2 0 log_to_monitors {default=true} 2021-03-11T00:42:24.658 INFO:tasks.ceph.c2.osd.1.smithi124.stderr:2021-03-11T00:42:24.652+0000 7f13b18f4d80 -1 osd.1 0 log_to_monitors {default=true} 2021-03-11T00:42:24.676 INFO:tasks.ceph.c2.osd.0.smithi124.stderr:2021-03-11T00:42:24.672+0000 7f1ab2183d80 -1 osd.0 0 log_to_monitors {default=true} 2021-03-11T00:42:26.452 INFO:tasks.ceph.c2.osd.2.smithi124.stderr:2021-03-11T00:42:26.448+0000 7f0faff3f700 -1 osd.2 0 waiting for initial osdmap 2021-03-11T00:42:26.453 INFO:tasks.ceph.c2.osd.1.smithi124.stderr:2021-03-11T00:42:26.448+0000 7f139b340700 -1 osd.1 0 waiting for initial osdmap 2021-03-11T00:42:26.453 INFO:tasks.ceph.c2.osd.0.smithi124.stderr:2021-03-11T00:42:26.448+0000 7f1a9bbcf700 -1 osd.0 0 waiting for initial osdmap 2021-03-11T00:42:30.245 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c2 osd dump --format=json 2021-03-11T00:42:30.516 INFO:teuthology.misc.health.smithi124.stdout: 2021-03-11T00:42:30.517 INFO:teuthology.misc.health.smithi124.stdout:{"epoch":10,"fsid":"80ab3993-6eec-4177-8595-ca0a327e3d06","created":"2021-03-11T00:42:22.411017+0000","modified":"2021-03-11T00:42:29.458812+0000","last_up_change":"2021-03-11T00:42:27.443780+0000","last_in_change":"2021-03-11T00:42:27.443780+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-03-11T00:42:26.376483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"df1c2a49-7081-4a2b-97f0-0bc5fc0c16c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":13520},{"type":"v1","addr":"172.21.15.124:6817","nonce":13520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":13520},{"type":"v1","addr":"172.21.15.124:6819","nonce":13520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":13520},{"type":"v1","addr":"172.21.15.124:6823","nonce":13520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":13520},{"type":"v1","addr":"172.21.15.124:6821","nonce":13520}]},"public_addr":"172.21.15.124:6817/13520","cluster_addr":"172.21.15.124:6819/13520","heartbeat_back_addr":"172.21.15.124:6823/13520","heartbeat_front_addr":"172.21.15.124:6821/13520","state":["exists","up"]},{"osd":1,"uuid":"1750f8ff-75de-41d9-829d-35fec51c6525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":8,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":13518},{"type":"v1","addr":"172.21.15.124:6808","nonce":13518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":13518},{"type":"v1","addr":"172.21.15.124:6811","nonce":13518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":13518},{"type":"v1","addr":"172.21.15.124:6815","nonce":13518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":13518},{"type":"v1","addr":"172.21.15.124:6813","nonce":13518}]},"public_addr":"172.21.15.124:6808/13518","cluster_addr":"172.21.15.124:6811/13518","heartbeat_back_addr":"172.21.15.124:6815/13518","heartbeat_front_addr":"172.21.15.124:6813/13518","state":["exists","up"]},{"osd":2,"uuid":"60760aca-cfcb-4f88-bcca-b5ce6b49b937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":13519},{"type":"v1","addr":"172.21.15.124:6801","nonce":13519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":13519},{"type":"v1","addr":"172.21.15.124:6803","nonce":13519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6807","nonce":13519},{"type":"v1","addr":"172.21.15.124:6809","nonce":13519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":13519},{"type":"v1","addr":"172.21.15.124:6805","nonce":13519}]},"public_addr":"172.21.15.124:6801/13519","cluster_addr":"172.21.15.124:6803/13519","heartbeat_back_addr":"172.21.15.124:6809/13519","heartbeat_front_addr":"172.21.15.124:6805/13519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-03-11T00:42:25.642549+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-03-11T00:42:25.643303+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-03-11T00:42:25.643637+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-03-11T00:42:30.528 DEBUG:teuthology.misc:3 of 3 OSDs are up 2021-03-11T00:42:30.529 INFO:tasks.ceph:Creating RBD pool 2021-03-11T00:42:30.529 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster c2 osd pool create rbd 8 2021-03-11T00:42:31.490 INFO:teuthology.orchestra.run.smithi124.stderr:pool 'rbd' created 2021-03-11T00:42:31.504 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster c2 osd pool application enable rbd rbd --yes-i-really-mean-it 2021-03-11T00:42:32.501 INFO:teuthology.orchestra.run.smithi124.stderr:enabled application 'rbd' on pool 'rbd' 2021-03-11T00:42:32.514 INFO:tasks.ceph:Starting mds daemons in cluster c2... 2021-03-11T00:42:32.515 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd dump --format=json 2021-03-11T00:42:32.516 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2021-03-11T00:42:32.781 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:32.782 INFO:teuthology.orchestra.run.smithi124.stdout:{"epoch":12,"fsid":"80ab3993-6eec-4177-8595-ca0a327e3d06","created":"2021-03-11T00:42:22.411017+0000","modified":"2021-03-11T00:42:32.478751+0000","last_up_change":"2021-03-11T00:42:27.443780+0000","last_in_change":"2021-03-11T00:42:27.443780+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-03-11T00:42:26.376483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-03-11T00:42:30.854326+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"df1c2a49-7081-4a2b-97f0-0bc5fc0c16c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":13520},{"type":"v1","addr":"172.21.15.124:6817","nonce":13520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":13520},{"type":"v1","addr":"172.21.15.124:6819","nonce":13520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":13520},{"type":"v1","addr":"172.21.15.124:6823","nonce":13520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":13520},{"type":"v1","addr":"172.21.15.124:6821","nonce":13520}]},"public_addr":"172.21.15.124:6817/13520","cluster_addr":"172.21.15.124:6819/13520","heartbeat_back_addr":"172.21.15.124:6823/13520","heartbeat_front_addr":"172.21.15.124:6821/13520","state":["exists","up"]},{"osd":1,"uuid":"1750f8ff-75de-41d9-829d-35fec51c6525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":13518},{"type":"v1","addr":"172.21.15.124:6808","nonce":13518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":13518},{"type":"v1","addr":"172.21.15.124:6811","nonce":13518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":13518},{"type":"v1","addr":"172.21.15.124:6815","nonce":13518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":13518},{"type":"v1","addr":"172.21.15.124:6813","nonce":13518}]},"public_addr":"172.21.15.124:6808/13518","cluster_addr":"172.21.15.124:6811/13518","heartbeat_back_addr":"172.21.15.124:6815/13518","heartbeat_front_addr":"172.21.15.124:6813/13518","state":["exists","up"]},{"osd":2,"uuid":"60760aca-cfcb-4f88-bcca-b5ce6b49b937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":13519},{"type":"v1","addr":"172.21.15.124:6801","nonce":13519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":13519},{"type":"v1","addr":"172.21.15.124:6803","nonce":13519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6807","nonce":13519},{"type":"v1","addr":"172.21.15.124:6809","nonce":13519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":13519},{"type":"v1","addr":"172.21.15.124:6805","nonce":13519}]},"public_addr":"172.21.15.124:6801/13519","cluster_addr":"172.21.15.124:6803/13519","heartbeat_back_addr":"172.21.15.124:6809/13519","heartbeat_front_addr":"172.21.15.124:6805/13519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-03-11T00:42:25.642549+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-03-11T00:42:25.643303+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-03-11T00:42:25.643637+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-03-11T00:42:32.794 INFO:tasks.ceph.ceph_manager.c2:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-03-11T00:42:26.376483+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '10', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-03-11T00:42:30.854326+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-03-11T00:42:32.794 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd pool get device_health_metrics pg_num 2021-03-11T00:42:33.068 INFO:teuthology.orchestra.run.smithi124.stdout:pg_num: 1 2021-03-11T00:42:33.082 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd pool get rbd pg_num 2021-03-11T00:42:33.351 INFO:teuthology.orchestra.run.smithi124.stdout:pg_num: 8 2021-03-11T00:42:33.364 INFO:tasks.ceph:Waiting until c2 daemons up and pgs clean... 2021-03-11T00:42:33.365 INFO:tasks.ceph.ceph_manager.c2:waiting for mgr available 2021-03-11T00:42:33.365 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 mgr dump --format=json 2021-03-11T00:42:33.680 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:33.689 INFO:teuthology.orchestra.run.smithi124.stdout:{"epoch":4,"active_gid":4098,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":13332},{"type":"v1","addr":"172.21.15.124:6825","nonce":13332}]},"active_addr":"172.21.15.124:6825/13332","active_change":"2021-03-11T00:42:26.291312+0000","active_mgr_features":4540138292840890367,"available":true,"standbys":[],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading remoto library:No module named 'remoto'","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.124:0","nonce":3632009569}]},{"addrvec":[{"type":"v2","addr":"172.21.15.124:0","nonce":394399335}]},{"addrvec":[{"type":"v2","addr":"172.21.15.124:0","nonce":2360473469}]}]}} 2021-03-11T00:42:33.713 INFO:tasks.ceph.ceph_manager.c2:mgr available! 2021-03-11T00:42:33.714 INFO:tasks.ceph.ceph_manager.c2:waiting for all up 2021-03-11T00:42:33.714 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd dump --format=json 2021-03-11T00:42:33.982 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:33.983 INFO:teuthology.orchestra.run.smithi124.stdout:{"epoch":13,"fsid":"80ab3993-6eec-4177-8595-ca0a327e3d06","created":"2021-03-11T00:42:22.411017+0000","modified":"2021-03-11T00:42:33.495507+0000","last_up_change":"2021-03-11T00:42:27.443780+0000","last_in_change":"2021-03-11T00:42:27.443780+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-03-11T00:42:26.376483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-03-11T00:42:30.854326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"df1c2a49-7081-4a2b-97f0-0bc5fc0c16c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":13520},{"type":"v1","addr":"172.21.15.124:6817","nonce":13520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":13520},{"type":"v1","addr":"172.21.15.124:6819","nonce":13520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":13520},{"type":"v1","addr":"172.21.15.124:6823","nonce":13520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":13520},{"type":"v1","addr":"172.21.15.124:6821","nonce":13520}]},"public_addr":"172.21.15.124:6817/13520","cluster_addr":"172.21.15.124:6819/13520","heartbeat_back_addr":"172.21.15.124:6823/13520","heartbeat_front_addr":"172.21.15.124:6821/13520","state":["exists","up"]},{"osd":1,"uuid":"1750f8ff-75de-41d9-829d-35fec51c6525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":13518},{"type":"v1","addr":"172.21.15.124:6808","nonce":13518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":13518},{"type":"v1","addr":"172.21.15.124:6811","nonce":13518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":13518},{"type":"v1","addr":"172.21.15.124:6815","nonce":13518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":13518},{"type":"v1","addr":"172.21.15.124:6813","nonce":13518}]},"public_addr":"172.21.15.124:6808/13518","cluster_addr":"172.21.15.124:6811/13518","heartbeat_back_addr":"172.21.15.124:6815/13518","heartbeat_front_addr":"172.21.15.124:6813/13518","state":["exists","up"]},{"osd":2,"uuid":"60760aca-cfcb-4f88-bcca-b5ce6b49b937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":13519},{"type":"v1","addr":"172.21.15.124:6801","nonce":13519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":13519},{"type":"v1","addr":"172.21.15.124:6803","nonce":13519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6807","nonce":13519},{"type":"v1","addr":"172.21.15.124:6809","nonce":13519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":13519},{"type":"v1","addr":"172.21.15.124:6805","nonce":13519}]},"public_addr":"172.21.15.124:6801/13519","cluster_addr":"172.21.15.124:6803/13519","heartbeat_back_addr":"172.21.15.124:6809/13519","heartbeat_front_addr":"172.21.15.124:6805/13519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-03-11T00:42:25.642549+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-03-11T00:42:25.643303+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-03-11T00:42:25.643637+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-03-11T00:42:33.996 INFO:tasks.ceph.ceph_manager.c2:all up! 2021-03-11T00:42:33.996 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd dump --format=json 2021-03-11T00:42:34.268 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:34.269 INFO:teuthology.orchestra.run.smithi124.stdout:{"epoch":13,"fsid":"80ab3993-6eec-4177-8595-ca0a327e3d06","created":"2021-03-11T00:42:22.411017+0000","modified":"2021-03-11T00:42:33.495507+0000","last_up_change":"2021-03-11T00:42:27.443780+0000","last_in_change":"2021-03-11T00:42:27.443780+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-03-11T00:42:26.376483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-03-11T00:42:30.854326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"df1c2a49-7081-4a2b-97f0-0bc5fc0c16c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":13520},{"type":"v1","addr":"172.21.15.124:6817","nonce":13520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":13520},{"type":"v1","addr":"172.21.15.124:6819","nonce":13520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":13520},{"type":"v1","addr":"172.21.15.124:6823","nonce":13520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":13520},{"type":"v1","addr":"172.21.15.124:6821","nonce":13520}]},"public_addr":"172.21.15.124:6817/13520","cluster_addr":"172.21.15.124:6819/13520","heartbeat_back_addr":"172.21.15.124:6823/13520","heartbeat_front_addr":"172.21.15.124:6821/13520","state":["exists","up"]},{"osd":1,"uuid":"1750f8ff-75de-41d9-829d-35fec51c6525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":13518},{"type":"v1","addr":"172.21.15.124:6808","nonce":13518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":13518},{"type":"v1","addr":"172.21.15.124:6811","nonce":13518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":13518},{"type":"v1","addr":"172.21.15.124:6815","nonce":13518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":13518},{"type":"v1","addr":"172.21.15.124:6813","nonce":13518}]},"public_addr":"172.21.15.124:6808/13518","cluster_addr":"172.21.15.124:6811/13518","heartbeat_back_addr":"172.21.15.124:6815/13518","heartbeat_front_addr":"172.21.15.124:6813/13518","state":["exists","up"]},{"osd":2,"uuid":"60760aca-cfcb-4f88-bcca-b5ce6b49b937","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":13519},{"type":"v1","addr":"172.21.15.124:6801","nonce":13519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":13519},{"type":"v1","addr":"172.21.15.124:6803","nonce":13519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6807","nonce":13519},{"type":"v1","addr":"172.21.15.124:6809","nonce":13519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":13519},{"type":"v1","addr":"172.21.15.124:6805","nonce":13519}]},"public_addr":"172.21.15.124:6801/13519","cluster_addr":"172.21.15.124:6803/13519","heartbeat_back_addr":"172.21.15.124:6809/13519","heartbeat_front_addr":"172.21.15.124:6805/13519","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-03-11T00:42:25.642549+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-03-11T00:42:25.643303+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-03-11T00:42:25.643637+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-03-11T00:42:34.282 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 tell osd.0 flush_pg_stats 2021-03-11T00:42:34.442 INFO:teuthology.orchestra.run.smithi124.stdout:34359738372 2021-03-11T00:42:34.443 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 tell osd.1 flush_pg_stats 2021-03-11T00:42:34.592 INFO:teuthology.orchestra.run.smithi124.stdout:34359738372 2021-03-11T00:42:34.592 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 tell osd.2 flush_pg_stats 2021-03-11T00:42:34.749 INFO:teuthology.orchestra.run.smithi124.stdout:34359738371 2021-03-11T00:42:34.750 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd last-stat-seq osd.0 2021-03-11T00:42:35.020 INFO:teuthology.orchestra.run.smithi124.stdout:34359738371 2021-03-11T00:42:35.038 INFO:tasks.ceph.ceph_manager.c2:need seq 34359738372 got 34359738371 for osd.0 2021-03-11T00:42:36.039 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd last-stat-seq osd.0 2021-03-11T00:42:36.325 INFO:teuthology.orchestra.run.smithi124.stdout:34359738371 2021-03-11T00:42:36.337 INFO:tasks.ceph.ceph_manager.c2:need seq 34359738372 got 34359738371 for osd.0 2021-03-11T00:42:37.338 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd last-stat-seq osd.0 2021-03-11T00:42:37.627 INFO:teuthology.orchestra.run.smithi124.stdout:34359738372 2021-03-11T00:42:37.636 INFO:tasks.ceph.ceph_manager.c2:need seq 34359738372 got 34359738372 for osd.0 2021-03-11T00:42:37.637 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd last-stat-seq osd.1 2021-03-11T00:42:37.914 INFO:teuthology.orchestra.run.smithi124.stdout:34359738372 2021-03-11T00:42:37.927 INFO:tasks.ceph.ceph_manager.c2:need seq 34359738372 got 34359738372 for osd.1 2021-03-11T00:42:37.927 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd last-stat-seq osd.2 2021-03-11T00:42:38.203 INFO:teuthology.orchestra.run.smithi124.stdout:34359738371 2021-03-11T00:42:38.216 INFO:tasks.ceph.ceph_manager.c2:need seq 34359738371 got 34359738371 for osd.2 2021-03-11T00:42:38.216 INFO:tasks.ceph.ceph_manager.c2:waiting for clean 2021-03-11T00:42:38.217 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 pg dump --format=json 2021-03-11T00:42:38.486 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:38.488 INFO:teuthology.orchestra.run.smithi124.stderr:dumped all 2021-03-11T00:42:38.499 INFO:teuthology.orchestra.run.smithi124.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2021-03-11T00:42:38.302765+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":3146072,"kb_used_data":320,"kb_used_omap":0,"kb_used_meta":3145728,"kb_avail":311426728,"statfs":{"total":322122547200,"available":318900969472,"internally_reserved":3221225472,"allocated":327680,"data_stored":92382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3221225472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.802026"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:32.508684+0000","last_change":"2021-03-11T00:42:32.505318+0000","last_active":"2021-03-11T00:42:32.508684+0000","last_peered":"2021-03-11T00:42:32.508684+0000","last_clean":"2021-03-11T00:42:32.508684+0000","last_became_active":"2021-03-11T00:42:32.504979+0000","last_became_peered":"2021-03-11T00:42:32.504979+0000","last_unstale":"2021-03-11T00:42:32.508684+0000","last_undegraded":"2021-03-11T00:42:32.508684+0000","last_fullsized":"2021-03-11T00:42:32.508684+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_clean_scrub_stamp":"2021-03-11T00:42:31.471729+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:32.508663+0000","last_change":"2021-03-11T00:42:32.505183+0000","last_active":"2021-03-11T00:42:32.508663+0000","last_peered":"2021-03-11T00:42:32.508663+0000","last_clean":"2021-03-11T00:42:32.508663+0000","last_became_active":"2021-03-11T00:42:32.504928+0000","last_became_peered":"2021-03-11T00:42:32.504928+0000","last_unstale":"2021-03-11T00:42:32.508663+0000","last_undegraded":"2021-03-11T00:42:32.508663+0000","last_fullsized":"2021-03-11T00:42:32.508663+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_clean_scrub_stamp":"2021-03-11T00:42:31.471729+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:32.511504+0000","last_change":"2021-03-11T00:42:32.506488+0000","last_active":"2021-03-11T00:42:32.511504+0000","last_peered":"2021-03-11T00:42:32.511504+0000","last_clean":"2021-03-11T00:42:32.511504+0000","last_became_active":"2021-03-11T00:42:32.506147+0000","last_became_peered":"2021-03-11T00:42:32.506147+0000","last_unstale":"2021-03-11T00:42:32.511504+0000","last_undegraded":"2021-03-11T00:42:32.511504+0000","last_fullsized":"2021-03-11T00:42:32.511504+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_clean_scrub_stamp":"2021-03-11T00:42:31.471729+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"8","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:32.504675+0000","last_change":"2021-03-11T00:42:32.504675+0000","last_active":"2021-03-11T00:42:32.504675+0000","last_peered":"2021-03-11T00:42:32.504675+0000","last_clean":"2021-03-11T00:42:32.504675+0000","last_became_active":"2021-03-11T00:42:32.503555+0000","last_became_peered":"2021-03-11T00:42:32.503555+0000","last_unstale":"2021-03-11T00:42:32.504675+0000","last_undegraded":"2021-03-11T00:42:32.504675+0000","last_fullsized":"2021-03-11T00:42:32.504675+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_clean_scrub_stamp":"2021-03-11T00:42:31.471729+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'1","reported_seq":"13","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:32.496560+0000","last_change":"2021-03-11T00:42:28.679901+0000","last_active":"2021-03-11T00:42:32.496560+0000","last_peered":"2021-03-11T00:42:32.496560+0000","last_clean":"2021-03-11T00:42:32.496560+0000","last_became_active":"2021-03-11T00:42:28.678979+0000","last_became_peered":"2021-03-11T00:42:28.678979+0000","last_unstale":"2021-03-11T00:42:32.496560+0000","last_undegraded":"2021-03-11T00:42:32.496560+0000","last_fullsized":"2021-03-11T00:42:32.496560+0000","mapping_epoch":8,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":9,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:26.432792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:26.432792+0000","last_clean_scrub_stamp":"2021-03-11T00:42:26.432792+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:32.511465+0000","last_change":"2021-03-11T00:42:32.506007+0000","last_active":"2021-03-11T00:42:32.511465+0000","last_peered":"2021-03-11T00:42:32.511465+0000","last_clean":"2021-03-11T00:42:32.511465+0000","last_became_active":"2021-03-11T00:42:32.505693+0000","last_became_peered":"2021-03-11T00:42:32.505693+0000","last_unstale":"2021-03-11T00:42:32.511465+0000","last_undegraded":"2021-03-11T00:42:32.511465+0000","last_fullsized":"2021-03-11T00:42:32.511465+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_clean_scrub_stamp":"2021-03-11T00:42:31.471729+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"8","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:32.504139+0000","last_change":"2021-03-11T00:42:32.504139+0000","last_active":"2021-03-11T00:42:32.504139+0000","last_peered":"2021-03-11T00:42:32.504139+0000","last_clean":"2021-03-11T00:42:32.504139+0000","last_became_active":"2021-03-11T00:42:32.503753+0000","last_became_peered":"2021-03-11T00:42:32.503753+0000","last_unstale":"2021-03-11T00:42:32.504139+0000","last_undegraded":"2021-03-11T00:42:32.504139+0000","last_fullsized":"2021-03-11T00:42:32.504139+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_clean_scrub_stamp":"2021-03-11T00:42:31.471729+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"8","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:32.504297+0000","last_change":"2021-03-11T00:42:32.504297+0000","last_active":"2021-03-11T00:42:32.504297+0000","last_peered":"2021-03-11T00:42:32.504297+0000","last_clean":"2021-03-11T00:42:32.504297+0000","last_became_active":"2021-03-11T00:42:32.503691+0000","last_became_peered":"2021-03-11T00:42:32.503691+0000","last_unstale":"2021-03-11T00:42:32.504297+0000","last_undegraded":"2021-03-11T00:42:32.504297+0000","last_fullsized":"2021-03-11T00:42:32.504297+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_clean_scrub_stamp":"2021-03-11T00:42:31.471729+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:32.505565+0000","last_change":"2021-03-11T00:42:32.501225+0000","last_active":"2021-03-11T00:42:32.505565+0000","last_peered":"2021-03-11T00:42:32.505565+0000","last_clean":"2021-03-11T00:42:32.505565+0000","last_became_active":"2021-03-11T00:42:32.500713+0000","last_became_peered":"2021-03-11T00:42:32.500713+0000","last_unstale":"2021-03-11T00:42:32.505565+0000","last_undegraded":"2021-03-11T00:42:32.505565+0000","last_fullsized":"2021-03-11T00:42:32.505565+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_clean_scrub_stamp":"2021-03-11T00:42:31.471729+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738371,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048688,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808912,"statfs":{"total":107374182400,"available":106300325888,"internally_reserved":1073741824,"allocated":106496,"data_stored":29623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048688,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808912,"statfs":{"total":107374182400,"available":106300325888,"internally_reserved":1073741824,"allocated":106496,"data_stored":29623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]}]},{"osd":1,"up_from":8,"seq":34359738372,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":33136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]}]}]}} 2021-03-11T00:42:38.501 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 pg dump --format=json 2021-03-11T00:42:38.765 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:38.768 INFO:teuthology.orchestra.run.smithi124.stderr:dumped all 2021-03-11T00:42:38.779 INFO:teuthology.orchestra.run.smithi124.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2021-03-11T00:42:38.302765+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":3146072,"kb_used_data":320,"kb_used_omap":0,"kb_used_meta":3145728,"kb_avail":311426728,"statfs":{"total":322122547200,"available":318900969472,"internally_reserved":3221225472,"allocated":327680,"data_stored":92382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3221225472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.802026"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:32.508684+0000","last_change":"2021-03-11T00:42:32.505318+0000","last_active":"2021-03-11T00:42:32.508684+0000","last_peered":"2021-03-11T00:42:32.508684+0000","last_clean":"2021-03-11T00:42:32.508684+0000","last_became_active":"2021-03-11T00:42:32.504979+0000","last_became_peered":"2021-03-11T00:42:32.504979+0000","last_unstale":"2021-03-11T00:42:32.508684+0000","last_undegraded":"2021-03-11T00:42:32.508684+0000","last_fullsized":"2021-03-11T00:42:32.508684+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_clean_scrub_stamp":"2021-03-11T00:42:31.471729+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:32.508663+0000","last_change":"2021-03-11T00:42:32.505183+0000","last_active":"2021-03-11T00:42:32.508663+0000","last_peered":"2021-03-11T00:42:32.508663+0000","last_clean":"2021-03-11T00:42:32.508663+0000","last_became_active":"2021-03-11T00:42:32.504928+0000","last_became_peered":"2021-03-11T00:42:32.504928+0000","last_unstale":"2021-03-11T00:42:32.508663+0000","last_undegraded":"2021-03-11T00:42:32.508663+0000","last_fullsized":"2021-03-11T00:42:32.508663+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_clean_scrub_stamp":"2021-03-11T00:42:31.471729+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:32.511504+0000","last_change":"2021-03-11T00:42:32.506488+0000","last_active":"2021-03-11T00:42:32.511504+0000","last_peered":"2021-03-11T00:42:32.511504+0000","last_clean":"2021-03-11T00:42:32.511504+0000","last_became_active":"2021-03-11T00:42:32.506147+0000","last_became_peered":"2021-03-11T00:42:32.506147+0000","last_unstale":"2021-03-11T00:42:32.511504+0000","last_undegraded":"2021-03-11T00:42:32.511504+0000","last_fullsized":"2021-03-11T00:42:32.511504+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_clean_scrub_stamp":"2021-03-11T00:42:31.471729+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"8","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:32.504675+0000","last_change":"2021-03-11T00:42:32.504675+0000","last_active":"2021-03-11T00:42:32.504675+0000","last_peered":"2021-03-11T00:42:32.504675+0000","last_clean":"2021-03-11T00:42:32.504675+0000","last_became_active":"2021-03-11T00:42:32.503555+0000","last_became_peered":"2021-03-11T00:42:32.503555+0000","last_unstale":"2021-03-11T00:42:32.504675+0000","last_undegraded":"2021-03-11T00:42:32.504675+0000","last_fullsized":"2021-03-11T00:42:32.504675+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_clean_scrub_stamp":"2021-03-11T00:42:31.471729+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"9'1","reported_seq":"13","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:32.496560+0000","last_change":"2021-03-11T00:42:28.679901+0000","last_active":"2021-03-11T00:42:32.496560+0000","last_peered":"2021-03-11T00:42:32.496560+0000","last_clean":"2021-03-11T00:42:32.496560+0000","last_became_active":"2021-03-11T00:42:28.678979+0000","last_became_peered":"2021-03-11T00:42:28.678979+0000","last_unstale":"2021-03-11T00:42:32.496560+0000","last_undegraded":"2021-03-11T00:42:32.496560+0000","last_fullsized":"2021-03-11T00:42:32.496560+0000","mapping_epoch":8,"log_start":"0'0","ondisk_log_start":"0'0","created":7,"last_epoch_clean":9,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:26.432792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:26.432792+0000","last_clean_scrub_stamp":"2021-03-11T00:42:26.432792+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:32.511465+0000","last_change":"2021-03-11T00:42:32.506007+0000","last_active":"2021-03-11T00:42:32.511465+0000","last_peered":"2021-03-11T00:42:32.511465+0000","last_clean":"2021-03-11T00:42:32.511465+0000","last_became_active":"2021-03-11T00:42:32.505693+0000","last_became_peered":"2021-03-11T00:42:32.505693+0000","last_unstale":"2021-03-11T00:42:32.511465+0000","last_undegraded":"2021-03-11T00:42:32.511465+0000","last_fullsized":"2021-03-11T00:42:32.511465+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_clean_scrub_stamp":"2021-03-11T00:42:31.471729+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"8","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:32.504139+0000","last_change":"2021-03-11T00:42:32.504139+0000","last_active":"2021-03-11T00:42:32.504139+0000","last_peered":"2021-03-11T00:42:32.504139+0000","last_clean":"2021-03-11T00:42:32.504139+0000","last_became_active":"2021-03-11T00:42:32.503753+0000","last_became_peered":"2021-03-11T00:42:32.503753+0000","last_unstale":"2021-03-11T00:42:32.504139+0000","last_undegraded":"2021-03-11T00:42:32.504139+0000","last_fullsized":"2021-03-11T00:42:32.504139+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_clean_scrub_stamp":"2021-03-11T00:42:31.471729+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"8","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:32.504297+0000","last_change":"2021-03-11T00:42:32.504297+0000","last_active":"2021-03-11T00:42:32.504297+0000","last_peered":"2021-03-11T00:42:32.504297+0000","last_clean":"2021-03-11T00:42:32.504297+0000","last_became_active":"2021-03-11T00:42:32.503691+0000","last_became_peered":"2021-03-11T00:42:32.503691+0000","last_unstale":"2021-03-11T00:42:32.504297+0000","last_undegraded":"2021-03-11T00:42:32.504297+0000","last_fullsized":"2021-03-11T00:42:32.504297+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_clean_scrub_stamp":"2021-03-11T00:42:31.471729+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"12","state":"active+clean","last_fresh":"2021-03-11T00:42:32.505565+0000","last_change":"2021-03-11T00:42:32.501225+0000","last_active":"2021-03-11T00:42:32.505565+0000","last_peered":"2021-03-11T00:42:32.505565+0000","last_clean":"2021-03-11T00:42:32.505565+0000","last_became_active":"2021-03-11T00:42:32.500713+0000","last_became_peered":"2021-03-11T00:42:32.500713+0000","last_unstale":"2021-03-11T00:42:32.505565+0000","last_undegraded":"2021-03-11T00:42:32.505565+0000","last_fullsized":"2021-03-11T00:42:32.505565+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-11T00:42:31.471729+0000","last_clean_scrub_stamp":"2021-03-11T00:42:31.471729+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738371,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048688,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808912,"statfs":{"total":107374182400,"available":106300325888,"internally_reserved":1073741824,"allocated":106496,"data_stored":29623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048688,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808912,"statfs":{"total":107374182400,"available":106300325888,"internally_reserved":1073741824,"allocated":106496,"data_stored":29623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]}]},{"osd":1,"up_from":8,"seq":34359738372,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":33136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]}]}]}} 2021-03-11T00:42:38.780 INFO:tasks.ceph.ceph_manager.c2:clean! 2021-03-11T00:42:38.780 INFO:tasks.ceph:Waiting until ceph cluster c2 is healthy... 2021-03-11T00:42:38.781 INFO:tasks.ceph.ceph_manager.c2:wait_until_healthy 2021-03-11T00:42:38.781 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 health --format=json 2021-03-11T00:42:39.105 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-03-11T00:42:39.118 INFO:teuthology.orchestra.run.smithi124.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2021-03-11T00:42:39.119 INFO:tasks.ceph.ceph_manager.c2:wait_until_healthy done 2021-03-11T00:42:39.119 INFO:teuthology.run_tasks:Running task parallel... 2021-03-11T00:42:39.133 INFO:teuthology.task.parallel:starting parallel... 2021-03-11T00:42:39.134 INFO:teuthology.task.parallel:In parallel, running task sequential... 2021-03-11T00:42:39.136 INFO:teuthology.task.sequential:In sequential, running task install.upgrade... 2021-03-11T00:42:39.136 INFO:teuthology.task.install:project ceph config {'c2.client.0': None, 'c2.client.1': None, 'c2.mgr.x': None, 'c2.mon.a': None, 'c2.osd.0': None, 'c2.osd.1': None, 'c2.osd.2': None} overrides {'branch': 'octopus', 'sha1': '409dc4b843da18e60e08a06ad16195db022fb227'} 2021-03-11T00:42:39.136 WARNING:teuthology.task.install:remote ubuntu@smithi124.front.sepia.ceph.com came up twice (role c2.client.1) 2021-03-11T00:42:39.137 WARNING:teuthology.task.install:remote ubuntu@smithi124.front.sepia.ceph.com came up twice (role c2.mgr.x) 2021-03-11T00:42:39.137 WARNING:teuthology.task.install:remote ubuntu@smithi124.front.sepia.ceph.com came up twice (role c2.mon.a) 2021-03-11T00:42:39.137 WARNING:teuthology.task.install:remote ubuntu@smithi124.front.sepia.ceph.com came up twice (role c2.osd.0) 2021-03-11T00:42:39.138 WARNING:teuthology.task.install:remote ubuntu@smithi124.front.sepia.ceph.com came up twice (role c2.osd.1) 2021-03-11T00:42:39.138 WARNING:teuthology.task.install:remote ubuntu@smithi124.front.sepia.ceph.com came up twice (role c2.osd.2) 2021-03-11T00:42:39.138 INFO:teuthology.task.install:remote ubuntu@smithi124.front.sepia.ceph.com config {'branch': 'octopus', 'sha1': '409dc4b843da18e60e08a06ad16195db022fb227'} 2021-03-11T00:42:39.139 INFO:teuthology.task.install:extra packages: [] 2021-03-11T00:42:39.139 DEBUG:teuthology.orchestra.run.smithi124:> sudo lsb_release -is 2021-03-11T00:42:39.201 INFO:teuthology.orchestra.run.smithi124.stdout:Ubuntu 2021-03-11T00:42:39.203 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-03-11T00:42:39.216 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2021-03-11T00:42:39.216 INFO:teuthology.task.install:Upgrading ceph deb packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse 2021-03-11T00:42:39.217 DEBUG:teuthology.orchestra.run.smithi124:> dpkg-query -W -f '${Version}' ceph-common 2021-03-11T00:42:39.247 INFO:teuthology.orchestra.run.smithi124.stdout:15.2.9-108-g1ff7d244-1bionic 2021-03-11T00:42:39.248 INFO:teuthology.packaging:The installed version of ceph-common is 15.2.9-108-g1ff7d244-1bionic 2021-03-11T00:42:39.249 DEBUG:teuthology.orchestra.run.smithi124:> sudo lsb_release -is 2021-03-11T00:42:39.343 INFO:teuthology.orchestra.run.smithi124.stdout:Ubuntu 2021-03-11T00:42:39.348 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-03-11T00:42:39.348 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2021-03-11T00:42:39.349 INFO:teuthology.packaging:ref: None 2021-03-11T00:42:39.349 INFO:teuthology.packaging:tag: None 2021-03-11T00:42:39.349 INFO:teuthology.packaging:branch: octopus 2021-03-11T00:42:39.350 INFO:teuthology.packaging:sha1: 409dc4b843da18e60e08a06ad16195db022fb227 2021-03-11T00:42:39.350 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=octopus 2021-03-11T00:42:39.567 INFO:teuthology.task.install:Ceph deb upgrade from 15.2.9-108-g1ff7d244-1bionic to 15.2.9-108-g1ff7d244-1bionic 2021-03-11T00:42:39.568 DEBUG:teuthology.orchestra.run.smithi124:> sudo apt-key list | grep Ceph 2021-03-11T00:42:39.680 INFO:teuthology.orchestra.run.smithi124.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2021-03-11T00:42:39.756 INFO:teuthology.orchestra.run.smithi124.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2021-03-11T00:42:39.756 INFO:teuthology.orchestra.run.smithi124.stdout:uid [ unknown] Ceph.com (release key) 2021-03-11T00:42:39.756 DEBUG:teuthology.orchestra.run.smithi124:> sudo lsb_release -is 2021-03-11T00:42:39.808 INFO:teuthology.orchestra.run.smithi124.stdout:Ubuntu 2021-03-11T00:42:39.813 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-03-11T00:42:39.813 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2021-03-11T00:42:39.814 INFO:teuthology.packaging:ref: None 2021-03-11T00:42:39.814 INFO:teuthology.packaging:tag: None 2021-03-11T00:42:39.814 INFO:teuthology.packaging:branch: octopus 2021-03-11T00:42:39.815 INFO:teuthology.packaging:sha1: 409dc4b843da18e60e08a06ad16195db022fb227 2021-03-11T00:42:39.815 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=octopus 2021-03-11T00:42:40.018 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default/ 2021-03-11T00:42:40.019 INFO:teuthology.task.install.deb:Package version is 15.2.9-108-g1ff7d244-1bionic 2021-03-11T00:42:40.331 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2021-03-11T00:42:40.332 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2021-03-11T00:42:40.348 DEBUG:teuthology.orchestra.run.smithi124:> sudo apt-get update 2021-03-11T00:42:40.528 INFO:teuthology.orchestra.run.smithi124.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2021-03-11T00:42:40.642 INFO:teuthology.orchestra.run.smithi124.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-03-11T00:42:40.726 INFO:teuthology.orchestra.run.smithi124.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2021-03-11T00:42:40.801 INFO:teuthology.orchestra.run.smithi124.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2021-03-11T00:42:40.818 INFO:teuthology.orchestra.run.smithi124.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic InRelease 2021-03-11T00:42:40.899 INFO:teuthology.orchestra.run.smithi124.stdout:Hit:6 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic Release 2021-03-11T00:42:40.974 INFO:teuthology.orchestra.run.smithi124.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/octopus/1ff7d244fcbc15312e010f47e19e0e96c819755d/ubuntu/bionic/flavors/default bionic Release.gpg 2021-03-11T00:42:42.368 INFO:teuthology.orchestra.run.smithi124.stdout:Reading package lists... 2021-03-11T00:42:42.415 DEBUG:teuthology.orchestra.run.smithi124:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.9-108-g1ff7d244-1bionic cephadm=15.2.9-108-g1ff7d244-1bionic ceph-mds=15.2.9-108-g1ff7d244-1bionic ceph-mgr=15.2.9-108-g1ff7d244-1bionic ceph-common=15.2.9-108-g1ff7d244-1bionic ceph-fuse=15.2.9-108-g1ff7d244-1bionic ceph-test=15.2.9-108-g1ff7d244-1bionic radosgw=15.2.9-108-g1ff7d244-1bionic python3-rados=15.2.9-108-g1ff7d244-1bionic python3-rgw=15.2.9-108-g1ff7d244-1bionic python3-cephfs=15.2.9-108-g1ff7d244-1bionic python3-rbd=15.2.9-108-g1ff7d244-1bionic libcephfs2=15.2.9-108-g1ff7d244-1bionic libcephfs-dev=15.2.9-108-g1ff7d244-1bionic librados2=15.2.9-108-g1ff7d244-1bionic librbd1=15.2.9-108-g1ff7d244-1bionic rbd-fuse=15.2.9-108-g1ff7d244-1bionic 2021-03-11T00:42:42.499 INFO:teuthology.orchestra.run.smithi124.stdout:Reading package lists... 2021-03-11T00:42:42.750 INFO:teuthology.orchestra.run.smithi124.stdout:Building dependency tree... 2021-03-11T00:42:42.751 INFO:teuthology.orchestra.run.smithi124.stdout:Reading state information... 2021-03-11T00:42:42.957 INFO:teuthology.orchestra.run.smithi124.stdout:ceph is already the newest version (15.2.9-108-g1ff7d244-1bionic). 2021-03-11T00:42:42.958 INFO:teuthology.orchestra.run.smithi124.stdout:ceph-common is already the newest version (15.2.9-108-g1ff7d244-1bionic). 2021-03-11T00:42:42.958 INFO:teuthology.orchestra.run.smithi124.stdout:ceph-fuse is already the newest version (15.2.9-108-g1ff7d244-1bionic). 2021-03-11T00:42:42.958 INFO:teuthology.orchestra.run.smithi124.stdout:ceph-mds is already the newest version (15.2.9-108-g1ff7d244-1bionic). 2021-03-11T00:42:42.959 INFO:teuthology.orchestra.run.smithi124.stdout:ceph-mgr is already the newest version (15.2.9-108-g1ff7d244-1bionic). 2021-03-11T00:42:42.960 INFO:teuthology.orchestra.run.smithi124.stdout:ceph-test is already the newest version (15.2.9-108-g1ff7d244-1bionic). 2021-03-11T00:42:42.960 INFO:teuthology.orchestra.run.smithi124.stdout:cephadm is already the newest version (15.2.9-108-g1ff7d244-1bionic). 2021-03-11T00:42:42.960 INFO:teuthology.orchestra.run.smithi124.stdout:libcephfs-dev is already the newest version (15.2.9-108-g1ff7d244-1bionic). 2021-03-11T00:42:42.960 INFO:teuthology.orchestra.run.smithi124.stdout:libcephfs2 is already the newest version (15.2.9-108-g1ff7d244-1bionic). 2021-03-11T00:42:42.961 INFO:teuthology.orchestra.run.smithi124.stdout:librados2 is already the newest version (15.2.9-108-g1ff7d244-1bionic). 2021-03-11T00:42:42.961 INFO:teuthology.orchestra.run.smithi124.stdout:librados2 set to manually installed. 2021-03-11T00:42:42.961 INFO:teuthology.orchestra.run.smithi124.stdout:librbd1 is already the newest version (15.2.9-108-g1ff7d244-1bionic). 2021-03-11T00:42:42.962 INFO:teuthology.orchestra.run.smithi124.stdout:librbd1 set to manually installed. 2021-03-11T00:42:42.962 INFO:teuthology.orchestra.run.smithi124.stdout:python3-cephfs is already the newest version (15.2.9-108-g1ff7d244-1bionic). 2021-03-11T00:42:42.962 INFO:teuthology.orchestra.run.smithi124.stdout:python3-rados is already the newest version (15.2.9-108-g1ff7d244-1bionic). 2021-03-11T00:42:42.963 INFO:teuthology.orchestra.run.smithi124.stdout:python3-rbd is already the newest version (15.2.9-108-g1ff7d244-1bionic). 2021-03-11T00:42:42.963 INFO:teuthology.orchestra.run.smithi124.stdout:python3-rgw is already the newest version (15.2.9-108-g1ff7d244-1bionic). 2021-03-11T00:42:42.963 INFO:teuthology.orchestra.run.smithi124.stdout:radosgw is already the newest version (15.2.9-108-g1ff7d244-1bionic). 2021-03-11T00:42:42.965 INFO:teuthology.orchestra.run.smithi124.stdout:rbd-fuse is already the newest version (15.2.9-108-g1ff7d244-1bionic). 2021-03-11T00:42:42.965 INFO:teuthology.orchestra.run.smithi124.stdout:0 upgraded, 0 newly installed, 0 to remove and 16 not upgraded. 2021-03-11T00:42:42.967 INFO:teuthology.orchestra.run.smithi124.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-03-11T00:42:42.967 DEBUG:teuthology.orchestra.run.smithi124:> sudo lsb_release -is 2021-03-11T00:42:43.015 INFO:teuthology.orchestra.run.smithi124.stdout:Ubuntu 2021-03-11T00:42:43.020 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-03-11T00:42:43.021 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2021-03-11T00:42:43.026 INFO:teuthology.packaging:ref: None 2021-03-11T00:42:43.027 INFO:teuthology.packaging:tag: None 2021-03-11T00:42:43.027 INFO:teuthology.packaging:branch: octopus 2021-03-11T00:42:43.027 INFO:teuthology.packaging:sha1: 409dc4b843da18e60e08a06ad16195db022fb227 2021-03-11T00:42:43.028 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=octopus 2021-03-11T00:42:43.238 DEBUG:teuthology.orchestra.run.smithi124:> dpkg-query -W -f '${Version}' ceph 2021-03-11T00:42:43.264 INFO:teuthology.orchestra.run.smithi124.stdout:15.2.9-108-g1ff7d244-1bionic 2021-03-11T00:42:43.266 INFO:teuthology.packaging:The installed version of ceph is 15.2.9-108-g1ff7d244-1bionic 2021-03-11T00:42:43.266 INFO:teuthology.task.install:The correct ceph version 15.2.9-108-g1ff7d244-1bionic is installed. 2021-03-11T00:42:43.267 DEBUG:teuthology.parallel:result is None 2021-03-11T00:42:43.267 INFO:teuthology.run_tasks:Running task rgw... 2021-03-11T00:42:43.291 DEBUG:tasks.rgw:config is {'c1.client.0': {'port': 8000}, 'c2.client.0': {'port': 8000}, 'c2.client.1': {'port': 8001}} 2021-03-11T00:42:43.291 DEBUG:tasks.rgw:client list is dict_keys(['c1.client.0', 'c2.client.0', 'c2.client.1']) 2021-03-11T00:42:43.292 INFO:tasks.rgw:Creating data pools 2021-03-11T00:42:43.292 DEBUG:tasks.rgw:Obtaining remote for client c1.client.0 2021-03-11T00:42:43.292 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph osd pool create default.rgw.buckets.data 64 64 --cluster c1 2021-03-11T00:42:43.697 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'default.rgw.buckets.data' created 2021-03-11T00:42:43.714 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster c1 2021-03-11T00:42:44.709 INFO:teuthology.orchestra.run.smithi119.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2021-03-11T00:42:44.740 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster c1 2021-03-11T00:42:45.718 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'default.rgw.buckets.index' created 2021-03-11T00:42:45.735 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster c1 2021-03-11T00:42:46.743 INFO:teuthology.orchestra.run.smithi119.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2021-03-11T00:42:46.759 DEBUG:tasks.rgw:Obtaining remote for client c2.client.0 2021-03-11T00:42:46.760 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph osd pool create default.rgw.buckets.data 64 64 --cluster c2 2021-03-11T00:42:47.420 INFO:teuthology.orchestra.run.smithi124.stderr:pool 'default.rgw.buckets.data' created 2021-03-11T00:42:47.433 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster c2 2021-03-11T00:42:48.454 INFO:teuthology.orchestra.run.smithi124.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2021-03-11T00:42:48.541 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster c2 2021-03-11T00:42:49.439 INFO:teuthology.orchestra.run.smithi124.stderr:pool 'default.rgw.buckets.index' created 2021-03-11T00:42:49.456 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster c2 2021-03-11T00:42:50.466 INFO:teuthology.orchestra.run.smithi124.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2021-03-11T00:42:50.486 DEBUG:tasks.rgw:Obtaining remote for client c2.client.1 2021-03-11T00:42:50.487 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph osd pool create default.rgw.buckets.data 64 64 --cluster c2 2021-03-11T00:42:50.753 INFO:teuthology.orchestra.run.smithi124.stderr:pool 'default.rgw.buckets.data' already exists 2021-03-11T00:42:50.763 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster c2 2021-03-11T00:42:51.469 INFO:teuthology.orchestra.run.smithi124.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2021-03-11T00:42:51.480 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster c2 2021-03-11T00:42:51.747 INFO:teuthology.orchestra.run.smithi124.stderr:pool 'default.rgw.buckets.index' already exists 2021-03-11T00:42:51.756 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster c2 2021-03-11T00:42:52.482 INFO:teuthology.orchestra.run.smithi124.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2021-03-11T00:42:52.496 DEBUG:tasks.rgw:Pools created 2021-03-11T00:42:52.496 INFO:tasks.rgw:Configuring compression type = random 2021-03-11T00:42:52.497 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['user', 'list'] 2021-03-11T00:42:52.497 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'user', 'list'] 2021-03-11T00:42:52.498 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 user list 2021-03-11T00:42:52.534 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:42:52.535 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:42:52.547 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:52.547+0000 7f29fc221080 20 rados->read ofs=0 len=0 2021-03-11T00:42:54.584 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.583+0000 7f29fc221080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:42:54.585 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.583+0000 7f29fc221080 20 rados->read ofs=0 len=0 2021-03-11T00:42:54.589 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.587+0000 7f29fc221080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:42:54.591 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.591+0000 7f29fc221080 20 rados->read ofs=0 len=0 2021-03-11T00:42:54.592 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.591+0000 7f29fc221080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:42:54.593 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.591+0000 7f29fc221080 20 rados->read ofs=0 len=0 2021-03-11T00:42:54.595 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.591+0000 7f29fc221080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:42:54.606 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.603+0000 7f29fc221080 20 rados->read ofs=0 len=0 2021-03-11T00:42:54.606 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.603+0000 7f29fc221080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:42:54.607 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.603+0000 7f29fc221080 20 rados->read ofs=0 len=0 2021-03-11T00:42:54.607 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.603+0000 7f29fc221080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:42:54.608 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.603+0000 7f29fc221080 20 rados->read ofs=0 len=0 2021-03-11T00:42:54.608 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.603+0000 7f29fc221080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:42:54.608 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.603+0000 7f29fc221080 10 cannot find current period zonegroup using local zonegroup 2021-03-11T00:42:54.609 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.603+0000 7f29fc221080 20 rados->read ofs=0 len=0 2021-03-11T00:42:54.609 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.603+0000 7f29fc221080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:42:54.609 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.607+0000 7f29fc221080 20 rados->read ofs=0 len=0 2021-03-11T00:42:54.610 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.607+0000 7f29fc221080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:42:54.610 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.607+0000 7f29fc221080 10 Creating default zonegroup 2021-03-11T00:42:54.611 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.607+0000 7f29fc221080 20 get_system_obj_state: rctx=0x7fffae6104e0 obj=:.pools.avail state=0x55e4d1a19220 s->prefetch_data=0 2021-03-11T00:42:54.612 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.607+0000 7f29fc221080 10 couldn't find old data placement pools config, setting up new ones for the zone 2021-03-11T00:42:54.619 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.619+0000 7f29fc221080 10 WARNING: store->list_zones() returned r=-2 2021-03-11T00:42:54.619 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.619+0000 7f29fc221080 20 rados->read ofs=0 len=0 2021-03-11T00:42:54.620 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.619+0000 7f29fc221080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:42:54.625 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.627+0000 7f29fc221080 20 rados->read ofs=0 len=0 2021-03-11T00:42:54.627 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.627+0000 7f29fc221080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:42:54.627 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.627+0000 7f29fc221080 10 could not read realm id: (2) No such file or directory 2021-03-11T00:42:54.628 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.627+0000 7f29fc221080 10 WARNING: failed to set zone as default, r=-22 2021-03-11T00:42:54.628 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.627+0000 7f29fc221080 20 rados->read ofs=0 len=0 2021-03-11T00:42:54.629 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.627+0000 7f29fc221080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:42:54.636 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.631+0000 7f29fc221080 20 rados->read ofs=0 len=0 2021-03-11T00:42:54.637 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.635+0000 7f29fc221080 20 rados_obj.operate() r=0 bl.length=839 2021-03-11T00:42:54.637 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.635+0000 7f29fc221080 20 rados->read ofs=0 len=0 2021-03-11T00:42:54.638 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.635+0000 7f29fc221080 20 rados_obj.operate() r=0 bl.length=358 2021-03-11T00:42:54.638 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.635+0000 7f29fc221080 20 zonegroup default 2021-03-11T00:42:54.638 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.635+0000 7f29fc221080 20 rados->read ofs=0 len=0 2021-03-11T00:42:54.639 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.635+0000 7f29fc221080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:42:54.639 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.635+0000 7f29fc221080 10 Cannot find current period zone using local zone 2021-03-11T00:42:54.639 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.635+0000 7f29fc221080 10 Using default name default 2021-03-11T00:42:54.640 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.635+0000 7f29fc221080 20 rados->read ofs=0 len=0 2021-03-11T00:42:54.640 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.635+0000 7f29fc221080 20 rados_obj.operate() r=0 bl.length=46 2021-03-11T00:42:54.640 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.635+0000 7f29fc221080 20 rados->read ofs=0 len=0 2021-03-11T00:42:54.641 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.635+0000 7f29fc221080 20 rados_obj.operate() r=0 bl.length=839 2021-03-11T00:42:54.641 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.635+0000 7f29fc221080 20 zone default found 2021-03-11T00:42:54.642 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:54.635+0000 7f29fc221080 20 rados->read ofs=0 len=0 2021-03-11T00:42:56.597 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:56.599+0000 7f29fc221080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:42:56.598 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:56.599+0000 7f29fc221080 20 rados->read ofs=0 len=0 2021-03-11T00:42:56.598 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:56.599+0000 7f29fc221080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:42:56.599 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:56.599+0000 7f29fc221080 20 started sync module instance, tier type = 2021-03-11T00:42:56.599 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:56.599+0000 7f29fc221080 20 started zone id=f29c9c70-bb1b-4c52-b420-836b8c567c39 (name=default) with tier type = 2021-03-11T00:42:58.659 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:58.659+0000 7f29fc221080 20 add_watcher() i=0 2021-03-11T00:42:58.660 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:58.659+0000 7f29fc221080 20 add_watcher() i=1 2021-03-11T00:42:58.660 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:58.659+0000 7f29fc221080 20 add_watcher() i=2 2021-03-11T00:42:58.660 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:58.659+0000 7f29fc221080 20 add_watcher() i=3 2021-03-11T00:42:58.661 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:58.659+0000 7f29fc221080 20 add_watcher() i=4 2021-03-11T00:42:58.661 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:58.659+0000 7f29fc221080 20 add_watcher() i=5 2021-03-11T00:42:58.661 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:58.659+0000 7f29fc221080 20 add_watcher() i=6 2021-03-11T00:42:58.662 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:58.659+0000 7f29fc221080 20 add_watcher() i=7 2021-03-11T00:42:58.663 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:58.659+0000 7f29fc221080 2 all 8 watchers are set, enabling cache 2021-03-11T00:42:58.663 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:58.659+0000 7f29fc221080 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2021-03-11T00:42:58.663 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:58.659+0000 7f29fc221080 20 check_secure_mon_conn(): mode 1 is insecure 2021-03-11T00:42:58.663 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:42:58.663+0000 7f29b97e2700 2 RGWDataChangesLog::ChangesRenewThread: start 2021-03-11T00:43:02.673 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:02.670+0000 7f29fc221080 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.meta 2021-03-11T00:43:02.773 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:02.770+0000 7f291f7fe700 20 reqs_thread_entry: start 2021-03-11T00:43:02.773 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:02.770+0000 7f291d7fa700 20 reqs_thread_entry: start 2021-03-11T00:43:02.774 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:02.770+0000 7f29fc221080 20 init_complete bucket index max shards: 11 2021-03-11T00:43:02.774 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:02.770+0000 7f290b7fe700 20 reqs_thread_entry: start 2021-03-11T00:43:02.788 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:02.786+0000 7f29b97e2700 2 RGWDataChangesLog::ChangesRenewThread: start 2021-03-11T00:43:02.788 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:02.786+0000 7f29fc221080 20 remove_watcher() i=0 2021-03-11T00:43:02.789 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:02.786+0000 7f29fc221080 2 removed watcher, disabling cache 2021-03-11T00:43:02.790 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:02.790+0000 7f29fc221080 20 remove_watcher() i=1 2021-03-11T00:43:02.793 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:02.790+0000 7f29fc221080 20 remove_watcher() i=2 2021-03-11T00:43:02.795 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:02.794+0000 7f29fc221080 20 remove_watcher() i=3 2021-03-11T00:43:02.797 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:02.798+0000 7f29fc221080 20 remove_watcher() i=4 2021-03-11T00:43:02.799 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:02.798+0000 7f29fc221080 20 remove_watcher() i=5 2021-03-11T00:43:02.801 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:02.802+0000 7f29fc221080 20 remove_watcher() i=6 2021-03-11T00:43:02.804 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:02.802+0000 7f29fc221080 20 remove_watcher() i=7 2021-03-11T00:43:02.811 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2021-03-11T00:43:02.812 DEBUG:tasks.util.rgw: json result: [] 2021-03-11T00:43:02.815 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2021-03-11T00:43:02.816 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2021-03-11T00:43:02.817 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone placement modify --rgw-zone default --placement-id default-placement --compression random 2021-03-11T00:43:02.895 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:43:02.895 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:43:02.905 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:02.902+0000 7fdea2d6c080 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2021-03-11T00:43:02.906 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:02.902+0000 7fdea2d6c080 20 check_secure_mon_conn(): mode 1 is insecure 2021-03-11T00:43:02.912 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:02.910+0000 7fddd77fe700 20 reqs_thread_entry: start 2021-03-11T00:43:02.912 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:02.910+0000 7fdea2d6c080 20 rados->read ofs=0 len=0 2021-03-11T00:43:02.913 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:02.914+0000 7fdea2d6c080 20 rados_obj.operate() r=0 bl.length=46 2021-03-11T00:43:02.913 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:02.914+0000 7fdea2d6c080 20 rados->read ofs=0 len=0 2021-03-11T00:43:02.915 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:02.914+0000 7fdea2d6c080 20 rados_obj.operate() r=0 bl.length=839 2021-03-11T00:43:02.915 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:02.914+0000 7fdea2d6c080 20 rados->read ofs=0 len=0 2021-03-11T00:43:02.915 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:02.914+0000 7fdea2d6c080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:43:02.915 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:02.914+0000 7fdea2d6c080 20 rados->read ofs=0 len=0 2021-03-11T00:43:02.916 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:02.914+0000 7fdea2d6c080 20 rados_obj.operate() r=0 bl.length=46 2021-03-11T00:43:02.916 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:02.914+0000 7fdea2d6c080 20 rados->read ofs=0 len=0 2021-03-11T00:43:02.917 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:02.914+0000 7fdea2d6c080 20 rados_obj.operate() r=0 bl.length=358 2021-03-11T00:43:02.941 INFO:teuthology.orchestra.run.smithi119.stdout:{"id":"f29c9c70-bb1b-4c52-b420-836b8c567c39","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"realm_id":""} 2021-03-11T00:43:02.942 DEBUG:tasks.util.rgw: json result: {'id': 'f29c9c70-bb1b-4c52-b420-836b8c567c39', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0}}], 'realm_id': ''} 2021-03-11T00:43:02.943 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['user', 'list'] 2021-03-11T00:43:02.943 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'user', 'list'] 2021-03-11T00:43:02.944 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 user list 2021-03-11T00:43:02.981 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:43:02.981 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:43:02.994 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:02.992+0000 7fe0dfe7b080 20 rados->read ofs=0 len=0 2021-03-11T00:43:04.425 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.420+0000 7fe0dfe7b080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:43:04.425 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.420+0000 7fe0dfe7b080 20 rados->read ofs=0 len=0 2021-03-11T00:43:04.432 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.428+0000 7fe0dfe7b080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:43:04.433 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.428+0000 7fe0dfe7b080 20 rados->read ofs=0 len=0 2021-03-11T00:43:04.433 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.432+0000 7fe0dfe7b080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:43:04.433 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.432+0000 7fe0dfe7b080 20 rados->read ofs=0 len=0 2021-03-11T00:43:04.436 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.432+0000 7fe0dfe7b080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:43:04.451 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.448+0000 7fe0dfe7b080 20 rados->read ofs=0 len=0 2021-03-11T00:43:04.451 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.448+0000 7fe0dfe7b080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:43:04.451 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.448+0000 7fe0dfe7b080 20 rados->read ofs=0 len=0 2021-03-11T00:43:04.452 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.448+0000 7fe0dfe7b080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:43:04.452 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.448+0000 7fe0dfe7b080 20 rados->read ofs=0 len=0 2021-03-11T00:43:04.452 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.448+0000 7fe0dfe7b080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:43:04.452 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.448+0000 7fe0dfe7b080 10 cannot find current period zonegroup using local zonegroup 2021-03-11T00:43:04.453 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.448+0000 7fe0dfe7b080 20 rados->read ofs=0 len=0 2021-03-11T00:43:04.453 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.448+0000 7fe0dfe7b080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:43:04.453 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.448+0000 7fe0dfe7b080 20 rados->read ofs=0 len=0 2021-03-11T00:43:04.454 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.448+0000 7fe0dfe7b080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:43:04.454 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.448+0000 7fe0dfe7b080 10 Creating default zonegroup 2021-03-11T00:43:04.455 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.448+0000 7fe0dfe7b080 20 get_system_obj_state: rctx=0x7fff74e75740 obj=:.pools.avail state=0x55e748dd8220 s->prefetch_data=0 2021-03-11T00:43:04.455 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.448+0000 7fe0dfe7b080 10 couldn't find old data placement pools config, setting up new ones for the zone 2021-03-11T00:43:04.462 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.460+0000 7fe0dfe7b080 10 WARNING: store->list_zones() returned r=-2 2021-03-11T00:43:04.462 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.460+0000 7fe0dfe7b080 20 rados->read ofs=0 len=0 2021-03-11T00:43:04.462 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.460+0000 7fe0dfe7b080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:43:04.466 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.464+0000 7fe0dfe7b080 20 rados->read ofs=0 len=0 2021-03-11T00:43:04.467 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.464+0000 7fe0dfe7b080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:43:04.467 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.464+0000 7fe0dfe7b080 10 could not read realm id: (2) No such file or directory 2021-03-11T00:43:04.467 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.464+0000 7fe0dfe7b080 10 WARNING: failed to set zone as default, r=-22 2021-03-11T00:43:04.467 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.464+0000 7fe0dfe7b080 20 rados->read ofs=0 len=0 2021-03-11T00:43:04.468 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.464+0000 7fe0dfe7b080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:43:04.472 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.472+0000 7fe0dfe7b080 20 rados->read ofs=0 len=0 2021-03-11T00:43:04.473 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.472+0000 7fe0dfe7b080 20 rados_obj.operate() r=0 bl.length=839 2021-03-11T00:43:04.474 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.472+0000 7fe0dfe7b080 20 rados->read ofs=0 len=0 2021-03-11T00:43:04.475 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.472+0000 7fe0dfe7b080 20 rados_obj.operate() r=0 bl.length=358 2021-03-11T00:43:04.475 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.472+0000 7fe0dfe7b080 20 zonegroup default 2021-03-11T00:43:04.475 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.472+0000 7fe0dfe7b080 20 rados->read ofs=0 len=0 2021-03-11T00:43:04.475 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.472+0000 7fe0dfe7b080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:43:04.476 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.472+0000 7fe0dfe7b080 10 Cannot find current period zone using local zone 2021-03-11T00:43:04.476 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.472+0000 7fe0dfe7b080 10 Using default name default 2021-03-11T00:43:04.476 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.472+0000 7fe0dfe7b080 20 rados->read ofs=0 len=0 2021-03-11T00:43:04.477 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.472+0000 7fe0dfe7b080 20 rados_obj.operate() r=0 bl.length=46 2021-03-11T00:43:04.477 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.472+0000 7fe0dfe7b080 20 rados->read ofs=0 len=0 2021-03-11T00:43:04.478 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.472+0000 7fe0dfe7b080 20 rados_obj.operate() r=0 bl.length=839 2021-03-11T00:43:04.478 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.472+0000 7fe0dfe7b080 20 zone default found 2021-03-11T00:43:04.478 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:04.472+0000 7fe0dfe7b080 20 rados->read ofs=0 len=0 2021-03-11T00:43:06.446 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:06.444+0000 7fe0dfe7b080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:43:06.446 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:06.444+0000 7fe0dfe7b080 20 rados->read ofs=0 len=0 2021-03-11T00:43:06.447 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:06.444+0000 7fe0dfe7b080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:43:06.447 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:06.444+0000 7fe0dfe7b080 20 started sync module instance, tier type = 2021-03-11T00:43:06.447 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:06.444+0000 7fe0dfe7b080 20 started zone id=c43f1cb9-4b2a-40c7-86d0-b43d8a7d42d1 (name=default) with tier type = 2021-03-11T00:43:08.507 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:08.500+0000 7fe0dfe7b080 20 add_watcher() i=0 2021-03-11T00:43:08.507 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:08.500+0000 7fe0dfe7b080 20 add_watcher() i=1 2021-03-11T00:43:08.508 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:08.500+0000 7fe0dfe7b080 20 add_watcher() i=2 2021-03-11T00:43:08.508 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:08.500+0000 7fe0dfe7b080 20 add_watcher() i=3 2021-03-11T00:43:08.508 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:08.500+0000 7fe0dfe7b080 20 add_watcher() i=4 2021-03-11T00:43:08.508 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:08.500+0000 7fe0dfe7b080 20 add_watcher() i=5 2021-03-11T00:43:08.509 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:08.500+0000 7fe0dfe7b080 20 add_watcher() i=6 2021-03-11T00:43:08.509 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:08.504+0000 7fe0dfe7b080 20 add_watcher() i=7 2021-03-11T00:43:08.509 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:08.504+0000 7fe0dfe7b080 2 all 8 watchers are set, enabling cache 2021-03-11T00:43:08.509 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:08.504+0000 7fe0dfe7b080 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2021-03-11T00:43:08.509 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:08.504+0000 7fe0dfe7b080 20 check_secure_mon_conn(): mode 1 is insecure 2021-03-11T00:43:08.510 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:08.504+0000 7fe09cfe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2021-03-11T00:43:12.473 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.472+0000 7fe0dfe7b080 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.meta 2021-03-11T00:43:12.576 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.572+0000 7fe00affd700 20 reqs_thread_entry: start 2021-03-11T00:43:12.577 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.572+0000 7fe0dfe7b080 20 init_complete bucket index max shards: 11 2021-03-11T00:43:12.577 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.572+0000 7fe008ff9700 20 reqs_thread_entry: start 2021-03-11T00:43:12.577 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.572+0000 7fdff6ffd700 20 reqs_thread_entry: start 2021-03-11T00:43:12.596 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.592+0000 7fe09cfe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2021-03-11T00:43:12.598 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.596+0000 7fe0dfe7b080 20 remove_watcher() i=0 2021-03-11T00:43:12.598 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.596+0000 7fe0dfe7b080 2 removed watcher, disabling cache 2021-03-11T00:43:12.600 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.596+0000 7fe0dfe7b080 20 remove_watcher() i=1 2021-03-11T00:43:12.602 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.600+0000 7fe0dfe7b080 20 remove_watcher() i=2 2021-03-11T00:43:12.605 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.604+0000 7fe0dfe7b080 20 remove_watcher() i=3 2021-03-11T00:43:12.612 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.604+0000 7fe0dfe7b080 20 remove_watcher() i=4 2021-03-11T00:43:12.613 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.608+0000 7fe0dfe7b080 20 remove_watcher() i=5 2021-03-11T00:43:12.613 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.608+0000 7fe0dfe7b080 20 remove_watcher() i=6 2021-03-11T00:43:12.615 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.612+0000 7fe0dfe7b080 20 remove_watcher() i=7 2021-03-11T00:43:12.622 INFO:teuthology.orchestra.run.smithi124.stdout:[] 2021-03-11T00:43:12.622 DEBUG:tasks.util.rgw: json result: [] 2021-03-11T00:43:12.623 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2021-03-11T00:43:12.623 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2021-03-11T00:43:12.624 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone placement modify --rgw-zone default --placement-id default-placement --compression random 2021-03-11T00:43:12.705 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:43:12.706 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:43:12.718 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.716+0000 7f63669b0080 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2021-03-11T00:43:12.719 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.716+0000 7f63669b0080 20 check_secure_mon_conn(): mode 1 is insecure 2021-03-11T00:43:12.728 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.724+0000 7f629b7fe700 20 reqs_thread_entry: start 2021-03-11T00:43:12.728 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.724+0000 7f63669b0080 20 rados->read ofs=0 len=0 2021-03-11T00:43:12.729 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.728+0000 7f63669b0080 20 rados_obj.operate() r=0 bl.length=46 2021-03-11T00:43:12.729 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.728+0000 7f63669b0080 20 rados->read ofs=0 len=0 2021-03-11T00:43:12.730 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.728+0000 7f63669b0080 20 rados_obj.operate() r=0 bl.length=839 2021-03-11T00:43:12.731 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.728+0000 7f63669b0080 20 rados->read ofs=0 len=0 2021-03-11T00:43:12.732 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.728+0000 7f63669b0080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:43:12.732 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.728+0000 7f63669b0080 20 rados->read ofs=0 len=0 2021-03-11T00:43:12.732 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.728+0000 7f63669b0080 20 rados_obj.operate() r=0 bl.length=46 2021-03-11T00:43:12.732 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.728+0000 7f63669b0080 20 rados->read ofs=0 len=0 2021-03-11T00:43:12.733 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.732+0000 7f63669b0080 20 rados_obj.operate() r=0 bl.length=358 2021-03-11T00:43:12.742 INFO:teuthology.orchestra.run.smithi124.stdout:{"id":"c43f1cb9-4b2a-40c7-86d0-b43d8a7d42d1","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"realm_id":""} 2021-03-11T00:43:12.743 DEBUG:tasks.util.rgw: json result: {'id': 'c43f1cb9-4b2a-40c7-86d0-b43d8a7d42d1', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0}}], 'realm_id': ''} 2021-03-11T00:43:12.743 INFO:tasks.util.rgw:rgwadmin: c2.client.1 : ['user', 'list'] 2021-03-11T00:43:12.743 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.1', '--cluster', 'c2', 'user', 'list'] 2021-03-11T00:43:12.744 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.1 --cluster c2 user list 2021-03-11T00:43:12.821 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:43:12.821 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:43:12.832 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.828+0000 7f951f5cb080 20 rados->read ofs=0 len=0 2021-03-11T00:43:12.834 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.832+0000 7f951f5cb080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:43:12.834 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.832+0000 7f951f5cb080 20 rados->read ofs=0 len=0 2021-03-11T00:43:12.835 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.832+0000 7f951f5cb080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:43:12.835 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.832+0000 7f951f5cb080 20 rados->read ofs=0 len=0 2021-03-11T00:43:12.835 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.832+0000 7f951f5cb080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:43:12.836 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.832+0000 7f951f5cb080 20 rados->read ofs=0 len=0 2021-03-11T00:43:12.836 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.832+0000 7f951f5cb080 20 rados_obj.operate() r=0 bl.length=46 2021-03-11T00:43:12.855 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.852+0000 7f951f5cb080 20 RGWRados::pool_iterate: got zone_info.c43f1cb9-4b2a-40c7-86d0-b43d8a7d42d1 2021-03-11T00:43:12.855 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.852+0000 7f951f5cb080 20 RGWRados::pool_iterate: got zonegroup_info.e5815c2c-0c1a-47cb-b579-9daba5060764 2021-03-11T00:43:12.855 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.852+0000 7f951f5cb080 20 RGWRados::pool_iterate: got zone_names.default 2021-03-11T00:43:12.855 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.852+0000 7f951f5cb080 20 RGWRados::pool_iterate: got zonegroups_names.default 2021-03-11T00:43:12.856 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.852+0000 7f951f5cb080 20 rados->read ofs=0 len=0 2021-03-11T00:43:12.856 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.852+0000 7f951f5cb080 20 rados_obj.operate() r=0 bl.length=46 2021-03-11T00:43:12.856 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.852+0000 7f951f5cb080 20 rados->read ofs=0 len=0 2021-03-11T00:43:12.858 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.852+0000 7f951f5cb080 20 rados_obj.operate() r=0 bl.length=855 2021-03-11T00:43:12.858 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.852+0000 7f951f5cb080 20 rados->read ofs=0 len=0 2021-03-11T00:43:12.858 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.852+0000 7f951f5cb080 20 rados_obj.operate() r=0 bl.length=46 2021-03-11T00:43:12.858 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.852+0000 7f951f5cb080 20 rados->read ofs=0 len=0 2021-03-11T00:43:12.859 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.856+0000 7f951f5cb080 20 rados_obj.operate() r=0 bl.length=358 2021-03-11T00:43:12.859 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.856+0000 7f951f5cb080 20 rados->read ofs=0 len=0 2021-03-11T00:43:12.859 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.856+0000 7f951f5cb080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:43:12.859 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.856+0000 7f951f5cb080 10 cannot find current period zonegroup using local zonegroup 2021-03-11T00:43:12.860 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.856+0000 7f951f5cb080 20 rados->read ofs=0 len=0 2021-03-11T00:43:12.860 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.856+0000 7f951f5cb080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:43:12.860 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.856+0000 7f951f5cb080 20 rados->read ofs=0 len=0 2021-03-11T00:43:12.861 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.856+0000 7f951f5cb080 20 rados_obj.operate() r=0 bl.length=46 2021-03-11T00:43:12.862 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.856+0000 7f951f5cb080 20 rados->read ofs=0 len=0 2021-03-11T00:43:12.862 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.856+0000 7f951f5cb080 20 rados_obj.operate() r=0 bl.length=358 2021-03-11T00:43:12.862 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.856+0000 7f951f5cb080 20 zonegroup default 2021-03-11T00:43:12.863 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.856+0000 7f951f5cb080 20 rados->read ofs=0 len=0 2021-03-11T00:43:12.863 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.856+0000 7f951f5cb080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:43:12.863 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.856+0000 7f951f5cb080 10 Cannot find current period zone using local zone 2021-03-11T00:43:12.863 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.856+0000 7f951f5cb080 20 rados->read ofs=0 len=0 2021-03-11T00:43:12.863 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.856+0000 7f951f5cb080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:43:12.864 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.856+0000 7f951f5cb080 20 rados->read ofs=0 len=0 2021-03-11T00:43:12.864 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.856+0000 7f951f5cb080 20 rados_obj.operate() r=0 bl.length=46 2021-03-11T00:43:12.864 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.856+0000 7f951f5cb080 20 rados->read ofs=0 len=0 2021-03-11T00:43:12.864 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.860+0000 7f951f5cb080 20 rados_obj.operate() r=0 bl.length=855 2021-03-11T00:43:12.865 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.860+0000 7f951f5cb080 20 zone default found 2021-03-11T00:43:12.865 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.860+0000 7f951f5cb080 20 rados->read ofs=0 len=0 2021-03-11T00:43:12.865 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.860+0000 7f951f5cb080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:43:12.865 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.860+0000 7f951f5cb080 20 rados->read ofs=0 len=0 2021-03-11T00:43:12.867 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.860+0000 7f951f5cb080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:43:12.867 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.860+0000 7f951f5cb080 20 started sync module instance, tier type = 2021-03-11T00:43:12.868 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.860+0000 7f951f5cb080 20 started zone id=c43f1cb9-4b2a-40c7-86d0-b43d8a7d42d1 (name=default) with tier type = 2021-03-11T00:43:12.893 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.888+0000 7f951f5cb080 20 add_watcher() i=0 2021-03-11T00:43:12.893 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.888+0000 7f951f5cb080 20 add_watcher() i=1 2021-03-11T00:43:12.894 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.888+0000 7f951f5cb080 20 add_watcher() i=2 2021-03-11T00:43:12.894 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.888+0000 7f951f5cb080 20 add_watcher() i=3 2021-03-11T00:43:12.894 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.888+0000 7f951f5cb080 20 add_watcher() i=4 2021-03-11T00:43:12.894 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.888+0000 7f951f5cb080 20 add_watcher() i=5 2021-03-11T00:43:12.894 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.892+0000 7f951f5cb080 20 add_watcher() i=6 2021-03-11T00:43:12.895 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.892+0000 7f951f5cb080 20 add_watcher() i=7 2021-03-11T00:43:12.896 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.892+0000 7f951f5cb080 2 all 8 watchers are set, enabling cache 2021-03-11T00:43:12.896 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.892+0000 7f951f5cb080 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2021-03-11T00:43:12.896 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.892+0000 7f951f5cb080 20 check_secure_mon_conn(): mode 1 is insecure 2021-03-11T00:43:12.896 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.892+0000 7f945ffff700 2 RGWDataChangesLog::ChangesRenewThread: start 2021-03-11T00:43:12.973 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.972+0000 7f944a7fc700 20 reqs_thread_entry: start 2021-03-11T00:43:12.974 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.972+0000 7f9443fff700 20 reqs_thread_entry: start 2021-03-11T00:43:12.975 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.972+0000 7f951f5cb080 20 init_complete bucket index max shards: 11 2021-03-11T00:43:12.975 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.972+0000 7f94427fc700 20 reqs_thread_entry: start 2021-03-11T00:43:12.996 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.992+0000 7f945ffff700 2 RGWDataChangesLog::ChangesRenewThread: start 2021-03-11T00:43:12.997 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.996+0000 7f951f5cb080 20 remove_watcher() i=0 2021-03-11T00:43:12.998 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.996+0000 7f951f5cb080 2 removed watcher, disabling cache 2021-03-11T00:43:13.000 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:12.996+0000 7f951f5cb080 20 remove_watcher() i=1 2021-03-11T00:43:13.002 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:13.000+0000 7f951f5cb080 20 remove_watcher() i=2 2021-03-11T00:43:13.004 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:13.000+0000 7f951f5cb080 20 remove_watcher() i=3 2021-03-11T00:43:13.006 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:13.004+0000 7f951f5cb080 20 remove_watcher() i=4 2021-03-11T00:43:13.009 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:13.008+0000 7f951f5cb080 20 remove_watcher() i=5 2021-03-11T00:43:13.011 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:13.008+0000 7f951f5cb080 20 remove_watcher() i=6 2021-03-11T00:43:13.013 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:13.012+0000 7f951f5cb080 20 remove_watcher() i=7 2021-03-11T00:43:13.022 INFO:teuthology.orchestra.run.smithi124.stdout:[] 2021-03-11T00:43:13.022 DEBUG:tasks.util.rgw: json result: [] 2021-03-11T00:43:13.023 INFO:tasks.util.rgw:rgwadmin: c2.client.1 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2021-03-11T00:43:13.023 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.1', '--cluster', 'c2', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2021-03-11T00:43:13.024 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.1 --cluster c2 zone placement modify --rgw-zone default --placement-id default-placement --compression random 2021-03-11T00:43:13.105 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:43:13.106 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:43:13.118 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:13.116+0000 7f07baffe080 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2021-03-11T00:43:13.118 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:13.116+0000 7f07baffe080 20 check_secure_mon_conn(): mode 1 is insecure 2021-03-11T00:43:13.125 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:13.120+0000 7f06ef7fe700 20 reqs_thread_entry: start 2021-03-11T00:43:13.125 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:13.124+0000 7f07baffe080 20 rados->read ofs=0 len=0 2021-03-11T00:43:13.126 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:13.124+0000 7f07baffe080 20 rados_obj.operate() r=0 bl.length=46 2021-03-11T00:43:13.126 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:13.124+0000 7f07baffe080 20 rados->read ofs=0 len=0 2021-03-11T00:43:13.127 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:13.124+0000 7f07baffe080 20 rados_obj.operate() r=0 bl.length=855 2021-03-11T00:43:13.128 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:13.124+0000 7f07baffe080 20 rados->read ofs=0 len=0 2021-03-11T00:43:13.128 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:13.124+0000 7f07baffe080 20 rados_obj.operate() r=-2 bl.length=0 2021-03-11T00:43:13.128 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:13.124+0000 7f07baffe080 20 rados->read ofs=0 len=0 2021-03-11T00:43:13.129 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:13.124+0000 7f07baffe080 20 rados_obj.operate() r=0 bl.length=46 2021-03-11T00:43:13.129 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:13.124+0000 7f07baffe080 20 rados->read ofs=0 len=0 2021-03-11T00:43:13.130 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:13.128+0000 7f07baffe080 20 rados_obj.operate() r=0 bl.length=358 2021-03-11T00:43:13.140 INFO:teuthology.orchestra.run.smithi124.stdout:{"id":"c43f1cb9-4b2a-40c7-86d0-b43d8a7d42d1","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"realm_id":""} 2021-03-11T00:43:13.141 DEBUG:tasks.util.rgw: json result: {'id': 'c43f1cb9-4b2a-40c7-86d0-b43d8a7d42d1', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0}}], 'realm_id': ''} 2021-03-11T00:43:13.141 INFO:tasks.rgw:Starting rgw... 2021-03-11T00:43:13.142 INFO:tasks.rgw:rgw c1.client.0 config is {'port': 8000} 2021-03-11T00:43:13.142 INFO:tasks.rgw:Using beast as radosgw frontend 2021-03-11T00:43:13.143 INFO:tasks.rgw.c1.client.0:Restarting daemon 2021-03-11T00:43:13.145 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8000' -n client.0 --cluster c1 -k /etc/ceph/c1.client.0.keyring --log-file /var/log/ceph/rgw.c1.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.c1.client.0.stdout 2>&1 2021-03-11T00:43:13.148 INFO:tasks.rgw.c1.client.0:Started 2021-03-11T00:43:13.148 INFO:tasks.rgw:rgw c2.client.0 config is {'port': 8000} 2021-03-11T00:43:13.148 INFO:tasks.rgw:Using beast as radosgw frontend 2021-03-11T00:43:13.149 INFO:tasks.rgw.c2.client.0:Restarting daemon 2021-03-11T00:43:13.149 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8000' -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.c2.client.0.stdout 2>&1 2021-03-11T00:43:13.182 INFO:tasks.rgw.c2.client.0:Started 2021-03-11T00:43:13.183 INFO:tasks.rgw:rgw c2.client.1 config is {'port': 8001} 2021-03-11T00:43:13.183 INFO:tasks.rgw:Using beast as radosgw frontend 2021-03-11T00:43:13.184 INFO:tasks.rgw.c2.client.1:Restarting daemon 2021-03-11T00:43:13.184 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8001' -n client.1 --cluster c2 -k /etc/ceph/c2.client.1.keyring --log-file /var/log/ceph/rgw.c2.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.1.sock --foreground | sudo tee /var/log/ceph/rgw.c2.client.1.stdout 2>&1 2021-03-11T00:43:13.187 INFO:tasks.rgw.c2.client.1:Started 2021-03-11T00:43:13.187 INFO:tasks.rgw:Polling c1.client.0 until it starts accepting connections on http://smithi119.front.sepia.ceph.com:8000/ 2021-03-11T00:43:13.188 DEBUG:teuthology.orchestra.run.smithi119:> curl http://smithi119.front.sepia.ceph.com:8000/ 2021-03-11T00:43:13.225 INFO:teuthology.orchestra.run.smithi119.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2021-03-11T00:43:13.225 INFO:teuthology.orchestra.run.smithi119.stderr: Dload Upload Total Spent Left Speed 2021-03-11T00:43:13.231 DEBUG:teuthology.orchestra.run:got remote process result: 7 2021-03-11T00:43:13.233 INFO:teuthology.orchestra.run.smithi119.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi119.front.sepia.ceph.com port 8000: Connection refused 2021-03-11T00:43:14.234 DEBUG:teuthology.orchestra.run.smithi119:> curl http://smithi119.front.sepia.ceph.com:8000/ 2021-03-11T00:43:14.254 INFO:teuthology.orchestra.run.smithi119.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2021-03-11T00:43:14.255 INFO:teuthology.orchestra.run.smithi119.stderr: Dload Upload Total Spent Left Speed 2021-03-11T00:43:14.258 INFO:teuthology.orchestra.run.smithi119.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 42800 0 --:--:-- --:--:-- --:--:-- 42800 2021-03-11T00:43:14.260 INFO:teuthology.orchestra.run.smithi119.stdout:anonymous 2021-03-11T00:43:14.261 INFO:tasks.rgw:Polling c2.client.0 until it starts accepting connections on http://smithi124.front.sepia.ceph.com:8000/ 2021-03-11T00:43:14.262 DEBUG:teuthology.orchestra.run.smithi124:> curl http://smithi124.front.sepia.ceph.com:8000/ 2021-03-11T00:43:14.507 INFO:teuthology.orchestra.run.smithi124.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2021-03-11T00:43:14.507 INFO:teuthology.orchestra.run.smithi124.stderr: Dload Upload Total Spent Left Speed 2021-03-11T00:43:14.513 INFO:teuthology.orchestra.run.smithi124.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 19454 0 --:--:-- --:--:-- --:--:-- 42800 2021-03-11T00:43:14.513 INFO:teuthology.orchestra.run.smithi124.stdout:anonymous 2021-03-11T00:43:14.513 INFO:tasks.rgw:Polling c2.client.1 until it starts accepting connections on http://smithi124.front.sepia.ceph.com:8001/ 2021-03-11T00:43:14.514 DEBUG:teuthology.orchestra.run.smithi124:> curl http://smithi124.front.sepia.ceph.com:8001/ 2021-03-11T00:43:14.524 INFO:teuthology.orchestra.run.smithi124.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2021-03-11T00:43:14.525 INFO:teuthology.orchestra.run.smithi124.stderr: Dload Upload Total Spent Left Speed 2021-03-11T00:43:14.530 INFO:teuthology.orchestra.run.smithi124.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 42800 0 --:--:-- --:--:-- --:--:-- 42800 2021-03-11T00:43:14.532 INFO:teuthology.orchestra.run.smithi124.stdout:anonymous 2021-03-11T00:43:14.532 INFO:teuthology.run_tasks:Running task rgw-multisite... 2021-03-11T00:43:15.000 INFO:tasks.rgw_multisite:creating realm.. 2021-03-11T00:43:15.001 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['realm', 'create', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:43:15.003 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'realm', 'create', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:43:15.004 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 realm create --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T00:43:15.042 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:43:15.043 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:43:15.089 INFO:teuthology.orchestra.run.smithi119.stdout:{"id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","name":"test-realm","current_period":"b521d673-4539-4378-9527-aedf408b14d1","epoch":1} 2021-03-11T00:43:15.089 INFO:tasks.rgw_multisite:creating master zonegroup.. 2021-03-11T00:43:15.090 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zonegroup', 'set', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:43:15.091 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zonegroup', 'set', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:43:15.092 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zonegroup set --rgw-zonegroup test-zonegroup --rgw-realm test-realm --default --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T00:43:15.170 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:43:15.171 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:43:15.193 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:43:15.186+0000 7f82647c8080 -1 old format 2021-03-11T00:43:15.208 INFO:teuthology.orchestra.run.smithi119.stdout:{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"","zones":[],"placement_targets":[],"default_placement":"","realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","sync_policy":{"groups":[]}} 2021-03-11T00:43:15.209 INFO:tasks.rgw_multisite:creating master zone.. 2021-03-11T00:43:15.209 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'create', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--master', '--access-key', '8WRXV4QK6AZ20SFM', '--secret', 'q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP', '--endpoints', 'http://smithi119.front.sepia.ceph.com:8000', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:43:15.210 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'create', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--master', '--access-key', '8WRXV4QK6AZ20SFM', '--secret', 'q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP', '--endpoints', 'http://smithi119.front.sepia.ceph.com:8000', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:43:15.210 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone create --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --default --master --access-key 8WRXV4QK6AZ20SFM --secret q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP --endpoints http://smithi119.front.sepia.ceph.com:8000 --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T00:43:15.296 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:43:15.296 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:43:15.364 INFO:teuthology.orchestra.run.smithi119.stdout:{"id":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","name":"test-zone1","domain_root":"test-zone1.rgw.meta:root","control_pool":"test-zone1.rgw.control","gc_pool":"test-zone1.rgw.log:gc","lc_pool":"test-zone1.rgw.log:lc","log_pool":"test-zone1.rgw.log","intent_log_pool":"test-zone1.rgw.log:intent","usage_log_pool":"test-zone1.rgw.log:usage","roles_pool":"test-zone1.rgw.meta:roles","reshard_pool":"test-zone1.rgw.log:reshard","user_keys_pool":"test-zone1.rgw.meta:users.keys","user_email_pool":"test-zone1.rgw.meta:users.email","user_swift_pool":"test-zone1.rgw.meta:users.swift","user_uid_pool":"test-zone1.rgw.meta:users.uid","otp_pool":"test-zone1.rgw.otp","system_key":{"access_key":"8WRXV4QK6AZ20SFM","secret_key":"q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone1.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone1.rgw.buckets.data"}},"data_extra_pool":"test-zone1.rgw.buckets.non-ec","index_type":0}}],"realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5"} 2021-03-11T00:43:15.365 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph osd pool create test-zone1.rgw.buckets.data 64 64 --cluster c1 2021-03-11T00:43:16.605 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'test-zone1.rgw.buckets.data' created 2021-03-11T00:43:16.618 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph osd pool application enable test-zone1.rgw.buckets.data rgw --cluster c1 2021-03-11T00:43:17.617 INFO:teuthology.orchestra.run.smithi119.stderr:enabled application 'rgw' on pool 'test-zone1.rgw.buckets.data' 2021-03-11T00:43:17.630 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'placement', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:43:17.631 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'placement', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:43:17.631 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone placement --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm modify --placement-id default-placement --compression random --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T00:43:17.666 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:43:17.666 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:43:17.691 INFO:teuthology.orchestra.run.smithi119.stdout:{"id":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","name":"test-zone1","domain_root":"test-zone1.rgw.meta:root","control_pool":"test-zone1.rgw.control","gc_pool":"test-zone1.rgw.log:gc","lc_pool":"test-zone1.rgw.log:lc","log_pool":"test-zone1.rgw.log","intent_log_pool":"test-zone1.rgw.log:intent","usage_log_pool":"test-zone1.rgw.log:usage","roles_pool":"test-zone1.rgw.meta:roles","reshard_pool":"test-zone1.rgw.log:reshard","user_keys_pool":"test-zone1.rgw.meta:users.keys","user_email_pool":"test-zone1.rgw.meta:users.email","user_swift_pool":"test-zone1.rgw.meta:users.swift","user_uid_pool":"test-zone1.rgw.meta:users.uid","otp_pool":"test-zone1.rgw.otp","system_key":{"access_key":"8WRXV4QK6AZ20SFM","secret_key":"q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone1.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone1.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone1.rgw.buckets.non-ec","index_type":0}}],"realm_id":""} 2021-03-11T00:43:17.692 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:43:17.693 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:43:17.694 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T00:43:17.770 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:43:17.770 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:43:25.840 INFO:teuthology.orchestra.run.smithi119.stdout:{"id":"470410ef-8c66-4339-a004-064d3e2c6c91","epoch":1,"predecessor_uuid":"b521d673-4539-4378-9527-aedf408b14d1","sync_status":[],"period_map":{"id":"470410ef-8c66-4339-a004-064d3e2c6c91","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","zones":[{"id":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","name":"test-zone1","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"false","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","sync_policy":{"groups":[]}}],"short_zone_ids":[{"key":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","val":3593284871}]},"master_zonegroup":"test-zonegroup","master_zone":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","realm_name":"test-realm","realm_epoch":2} 2021-03-11T00:43:25.841 DEBUG:tasks.rgw.c1.client.0:waiting for process to exit 2021-03-11T00:43:25.841 INFO:teuthology.orchestra.run:waiting for 300 2021-03-11T00:43:25.885 INFO:tasks.rgw.c1.client.0.smithi119.stdout:2021-03-11T00:43:25.882+0000 7f94f7fff700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term radosgw --rgw-frontends beast port=8000 -n client.0 --cluster c1 -k /etc/ceph/c1.client.0.keyring --log-file /var/log/ceph/rgw.c1.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.0.sock --foreground (PID: 14841) UID: 0 2021-03-11T00:43:25.886 INFO:tasks.rgw.c1.client.0.smithi119.stdout:2021-03-11T00:43:25.882+0000 7f951bd8f0c0 -1 shutting down 2021-03-11T00:43:27.983 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c1.rgw.client.0 is failed for ~0s 2021-03-11T00:43:31.944 INFO:tasks.rgw.c1.client.0:Stopped 2021-03-11T00:43:31.955 INFO:tasks.rgw.c1.client.0:Restarting daemon 2021-03-11T00:43:31.956 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8000' -n client.0 --cluster c1 -k /etc/ceph/c1.client.0.keyring --log-file /var/log/ceph/rgw.c1.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.0.sock --foreground --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c1.client.0.stdout 2>&1 2021-03-11T00:43:31.958 INFO:tasks.rgw.c1.client.0:Started 2021-03-11T00:43:31.959 DEBUG:teuthology.orchestra.run.smithi119:> curl http://smithi119.front.sepia.ceph.com:8000 2021-03-11T00:43:31.974 INFO:teuthology.orchestra.run.smithi119.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2021-03-11T00:43:31.975 INFO:teuthology.orchestra.run.smithi119.stderr: Dload Upload Total Spent Left Speed 2021-03-11T00:43:31.978 INFO:teuthology.orchestra.run.smithi119.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi119.front.sepia.ceph.com port 8000: Connection refused 2021-03-11T00:43:31.979 DEBUG:teuthology.orchestra.run:got remote process result: 7 2021-03-11T00:43:32.980 DEBUG:teuthology.orchestra.run.smithi119:> curl http://smithi119.front.sepia.ceph.com:8000 2021-03-11T00:43:32.998 INFO:teuthology.orchestra.run.smithi119.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2021-03-11T00:43:32.999 INFO:teuthology.orchestra.run.smithi119.stderr: Dload Upload Total Spent Left Speed 2021-03-11T00:43:33.009 INFO:teuthology.orchestra.run.smithi119.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 42800 0 --:--:-- --:--:-- --:--:-- 42800 2021-03-11T00:43:33.009 INFO:teuthology.orchestra.run.smithi119.stdout:anonymous 2021-03-11T00:43:33.009 INFO:tasks.rgw_multisite:creating admin user.. 2021-03-11T00:43:33.010 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['user', 'create', '--uid', 'realm-admin', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--display-name', 'Realm Admin', '--system', '--access-key', '8WRXV4QK6AZ20SFM', '--secret', 'q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:43:33.010 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'user', 'create', '--uid', 'realm-admin', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--display-name', 'Realm Admin', '--system', '--access-key', '8WRXV4QK6AZ20SFM', '--secret', 'q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:43:33.011 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 user create --uid realm-admin --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --display-name 'Realm Admin' --system --access-key 8WRXV4QK6AZ20SFM --secret q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T00:43:33.098 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:43:33.098 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:43:33.259 INFO:teuthology.orchestra.run.smithi119.stdout:{"user_id":"realm-admin","display_name":"Realm Admin","email":"","suspended":0,"max_buckets":1000,"subusers":[],"keys":[{"user":"realm-admin","access_key":"8WRXV4QK6AZ20SFM","secret_key":"q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","system":"true","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2021-03-11T00:43:33.284 INFO:tasks.rgw_multisite:pulling realm configuration to c2 2021-03-11T00:43:33.285 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['realm', 'pull', '--rgw-realm', 'test-realm', '--url', 'http://smithi119.front.sepia.ceph.com:8000', '--access-key', '8WRXV4QK6AZ20SFM', '--secret', 'q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:43:33.285 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'realm', 'pull', '--rgw-realm', 'test-realm', '--url', 'http://smithi119.front.sepia.ceph.com:8000', '--access-key', '8WRXV4QK6AZ20SFM', '--secret', 'q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:43:33.286 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 realm pull --rgw-realm test-realm --url http://smithi119.front.sepia.ceph.com:8000 --access-key 8WRXV4QK6AZ20SFM --secret q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP --cluster c2 --debug-rgw 0 --debug-ms 0 2021-03-11T00:43:33.333 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:43:33.333 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:43:33.438 INFO:teuthology.orchestra.run.smithi124.stdout:{"id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","name":"test-realm","current_period":"470410ef-8c66-4339-a004-064d3e2c6c91","epoch":2} 2021-03-11T00:43:33.439 INFO:tasks.rgw_multisite:creating zone.. 2021-03-11T00:43:33.440 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'create', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--access-key', '8WRXV4QK6AZ20SFM', '--secret', 'q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP', '--endpoints', 'http://smithi124.front.sepia.ceph.com:8000', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:43:33.441 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'create', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--access-key', '8WRXV4QK6AZ20SFM', '--secret', 'q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP', '--endpoints', 'http://smithi124.front.sepia.ceph.com:8000', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:43:33.442 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone create --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --default --access-key 8WRXV4QK6AZ20SFM --secret q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP --endpoints http://smithi124.front.sepia.ceph.com:8000 --cluster c2 --debug-rgw 0 --debug-ms 0 2021-03-11T00:43:33.525 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:43:33.525 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:43:33.578 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:33.569+0000 7f3dba180080 0 failed reading obj info from .rgw.root:zone_info.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0: (2) No such file or directory 2021-03-11T00:43:33.578 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:33.569+0000 7f3dba180080 0 WARNING: could not read zone params for zone id=4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0 name=test-zone1 2021-03-11T00:43:33.589 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c1.rgw.client.0 has been restored 2021-03-11T00:43:33.590 INFO:teuthology.orchestra.run.smithi124.stdout:{"id":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","name":"test-zone2","domain_root":"test-zone2.rgw.meta:root","control_pool":"test-zone2.rgw.control","gc_pool":"test-zone2.rgw.log:gc","lc_pool":"test-zone2.rgw.log:lc","log_pool":"test-zone2.rgw.log","intent_log_pool":"test-zone2.rgw.log:intent","usage_log_pool":"test-zone2.rgw.log:usage","roles_pool":"test-zone2.rgw.meta:roles","reshard_pool":"test-zone2.rgw.log:reshard","user_keys_pool":"test-zone2.rgw.meta:users.keys","user_email_pool":"test-zone2.rgw.meta:users.email","user_swift_pool":"test-zone2.rgw.meta:users.swift","user_uid_pool":"test-zone2.rgw.meta:users.uid","otp_pool":"test-zone2.rgw.otp","system_key":{"access_key":"8WRXV4QK6AZ20SFM","secret_key":"q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone2.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone2.rgw.buckets.data"}},"data_extra_pool":"test-zone2.rgw.buckets.non-ec","index_type":0}}],"realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5"} 2021-03-11T00:43:33.591 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph osd pool create test-zone2.rgw.buckets.data 64 64 --cluster c2 2021-03-11T00:43:34.452 INFO:teuthology.orchestra.run.smithi124.stderr:pool 'test-zone2.rgw.buckets.data' created 2021-03-11T00:43:34.470 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph osd pool application enable test-zone2.rgw.buckets.data rgw --cluster c2 2021-03-11T00:43:35.459 INFO:teuthology.orchestra.run.smithi124.stderr:enabled application 'rgw' on pool 'test-zone2.rgw.buckets.data' 2021-03-11T00:43:35.476 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'placement', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:43:35.486 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'placement', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:43:35.487 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone placement --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm modify --placement-id default-placement --compression random --cluster c2 --debug-rgw 0 --debug-ms 0 2021-03-11T00:43:35.520 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:43:35.521 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:43:35.550 INFO:teuthology.orchestra.run.smithi124.stdout:{"id":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","name":"test-zone2","domain_root":"test-zone2.rgw.meta:root","control_pool":"test-zone2.rgw.control","gc_pool":"test-zone2.rgw.log:gc","lc_pool":"test-zone2.rgw.log:lc","log_pool":"test-zone2.rgw.log","intent_log_pool":"test-zone2.rgw.log:intent","usage_log_pool":"test-zone2.rgw.log:usage","roles_pool":"test-zone2.rgw.meta:roles","reshard_pool":"test-zone2.rgw.log:reshard","user_keys_pool":"test-zone2.rgw.meta:users.keys","user_email_pool":"test-zone2.rgw.meta:users.email","user_swift_pool":"test-zone2.rgw.meta:users.swift","user_uid_pool":"test-zone2.rgw.meta:users.uid","otp_pool":"test-zone2.rgw.otp","system_key":{"access_key":"8WRXV4QK6AZ20SFM","secret_key":"q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone2.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone2.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone2.rgw.buckets.non-ec","index_type":0}}],"realm_id":""} 2021-03-11T00:43:35.551 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:43:35.551 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:43:35.551 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2021-03-11T00:43:35.629 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:43:35.629 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:43:43.633 INFO:teuthology.orchestra.run.smithi124.stderr:Sending period to new master zone 4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0 2021-03-11T00:43:44.712 INFO:teuthology.orchestra.run.smithi124.stdout:{"id":"470410ef-8c66-4339-a004-064d3e2c6c91","epoch":2,"predecessor_uuid":"b521d673-4539-4378-9527-aedf408b14d1","sync_status":[],"period_map":{"id":"470410ef-8c66-4339-a004-064d3e2c6c91","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","zones":[{"id":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","name":"test-zone1","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","name":"test-zone2","endpoints":["http://smithi124.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","sync_policy":{"groups":[]}}],"short_zone_ids":[{"key":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","val":3593284871},{"key":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","val":1094358664}]},"master_zonegroup":"test-zonegroup","master_zone":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","realm_name":"test-realm","realm_epoch":2} 2021-03-11T00:43:44.715 DEBUG:tasks.rgw.c2.client.0:waiting for process to exit 2021-03-11T00:43:44.720 INFO:teuthology.orchestra.run:waiting for 300 2021-03-11T00:43:44.756 INFO:tasks.rgw.c2.client.0.smithi124.stdout:2021-03-11T00:43:44.753+0000 7f143a7fc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term radosgw --rgw-frontends beast port=8000 -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground (PID: 15439) UID: 0 2021-03-11T00:43:44.756 INFO:tasks.rgw.c2.client.0.smithi124.stdout:2021-03-11T00:43:44.753+0000 7f14560980c0 -1 shutting down 2021-03-11T00:43:45.698 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~0s 2021-03-11T00:43:50.824 INFO:tasks.rgw.c2.client.0:Stopped 2021-03-11T00:43:50.825 INFO:tasks.rgw.c2.client.0:Restarting daemon 2021-03-11T00:43:50.827 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8000' -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c2.client.0.stdout 2>&1 2021-03-11T00:43:50.831 INFO:tasks.rgw.c2.client.0:Started 2021-03-11T00:43:50.832 DEBUG:teuthology.orchestra.run.smithi124:> curl http://smithi124.front.sepia.ceph.com:8000 2021-03-11T00:43:50.851 INFO:teuthology.orchestra.run.smithi124.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2021-03-11T00:43:50.852 INFO:teuthology.orchestra.run.smithi124.stderr: Dload Upload Total Spent Left Speed 2021-03-11T00:43:50.854 DEBUG:teuthology.orchestra.run:got remote process result: 7 2021-03-11T00:43:50.856 INFO:teuthology.orchestra.run.smithi124.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi124.front.sepia.ceph.com port 8000: Connection refused 2021-03-11T00:43:51.406 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 has been restored 2021-03-11T00:43:51.857 DEBUG:teuthology.orchestra.run.smithi124:> curl http://smithi124.front.sepia.ceph.com:8000 2021-03-11T00:43:51.884 INFO:teuthology.orchestra.run.smithi124.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2021-03-11T00:43:51.884 INFO:teuthology.orchestra.run.smithi124.stderr: Dload Upload Total Spent Left Speed 2021-03-11T00:43:51.888 INFO:teuthology.orchestra.run.smithi124.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 42800 0 --:--:-- --:--:-- --:--:-- 42800 2021-03-11T00:43:51.889 INFO:teuthology.orchestra.run.smithi124.stdout:anonymous 2021-03-11T00:43:51.890 INFO:tasks.rgw_multisite:pulling realm configuration to c2 2021-03-11T00:43:51.890 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['realm', 'pull', '--rgw-realm', 'test-realm', '--url', 'http://smithi119.front.sepia.ceph.com:8000', '--access-key', '8WRXV4QK6AZ20SFM', '--secret', 'q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP', '--url', 'http://smithi119.front.sepia.ceph.com:8000', '--access-key', '8WRXV4QK6AZ20SFM', '--secret', 'q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:43:51.891 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'realm', 'pull', '--rgw-realm', 'test-realm', '--url', 'http://smithi119.front.sepia.ceph.com:8000', '--access-key', '8WRXV4QK6AZ20SFM', '--secret', 'q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP', '--url', 'http://smithi119.front.sepia.ceph.com:8000', '--access-key', '8WRXV4QK6AZ20SFM', '--secret', 'q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:43:51.891 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 realm pull --rgw-realm test-realm --url http://smithi119.front.sepia.ceph.com:8000 --access-key 8WRXV4QK6AZ20SFM --secret q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP --url http://smithi119.front.sepia.ceph.com:8000 --access-key 8WRXV4QK6AZ20SFM --secret q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP --cluster c2 --debug-rgw 0 --debug-ms 0 2021-03-11T00:43:51.980 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:43:51.981 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:43:52.023 INFO:tasks.rgw.c2.client.0.smithi124.stdout:2021-03-11T00:43:52.017+0000 7fd0b4620700 -1 rgw realm watcher: RGWRealmWatcher::handle_error oid=realms.8cdef83d-c2ff-437f-989e-15e4c59256d5.control err=-107 2021-03-11T00:43:52.045 INFO:teuthology.orchestra.run.smithi124.stdout:{"id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","name":"test-realm","current_period":"470410ef-8c66-4339-a004-064d3e2c6c91","epoch":2} 2021-03-11T00:43:52.045 INFO:tasks.rgw_multisite:creating zone.. 2021-03-11T00:43:52.046 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'create', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--access-key', '8WRXV4QK6AZ20SFM', '--secret', 'q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP', '--endpoints', 'http://smithi124.front.sepia.ceph.com:8001', '--tier-type', 'pubsub', '--sync-from-all=0', '--sync-from', 'test-zone1', '--tier-config', 'start_with_full_sync=false,event_retention_days=7', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:43:52.048 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'create', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--access-key', '8WRXV4QK6AZ20SFM', '--secret', 'q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP', '--endpoints', 'http://smithi124.front.sepia.ceph.com:8001', '--tier-type', 'pubsub', '--sync-from-all=0', '--sync-from', 'test-zone1', '--tier-config', 'start_with_full_sync=false,event_retention_days=7', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:43:52.048 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone create --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --access-key 8WRXV4QK6AZ20SFM --secret q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP --endpoints http://smithi124.front.sepia.ceph.com:8001 --tier-type pubsub --sync-from-all=0 --sync-from test-zone1 --tier-config start_with_full_sync=false,event_retention_days=7 --cluster c2 --debug-rgw 0 --debug-ms 0 2021-03-11T00:43:52.100 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:43:52.101 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:43:52.156 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:52.149+0000 7fc81962a080 0 failed reading obj info from .rgw.root:zone_info.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0: (2) No such file or directory 2021-03-11T00:43:52.156 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:43:52.149+0000 7fc81962a080 0 WARNING: could not read zone params for zone id=4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0 name=test-zone1 2021-03-11T00:43:52.166 INFO:teuthology.orchestra.run.smithi124.stdout:{"id":"2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05","name":"test-zone3","domain_root":"test-zone3.rgw.meta:root","control_pool":"test-zone3.rgw.control","gc_pool":"test-zone3.rgw.log:gc","lc_pool":"test-zone3.rgw.log:lc","log_pool":"test-zone3.rgw.log","intent_log_pool":"test-zone3.rgw.log:intent","usage_log_pool":"test-zone3.rgw.log:usage","roles_pool":"test-zone3.rgw.meta:roles","reshard_pool":"test-zone3.rgw.log:reshard","user_keys_pool":"test-zone3.rgw.meta:users.keys","user_email_pool":"test-zone3.rgw.meta:users.email","user_swift_pool":"test-zone3.rgw.meta:users.swift","user_uid_pool":"test-zone3.rgw.meta:users.uid","otp_pool":"test-zone3.rgw.otp","system_key":{"access_key":"8WRXV4QK6AZ20SFM","secret_key":"q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone3.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone3.rgw.buckets.data"}},"data_extra_pool":"test-zone3.rgw.buckets.non-ec","index_type":0}}],"tier_config":{"event_retention_days":7,"start_with_full_sync":false},"realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5"} 2021-03-11T00:43:52.167 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph osd pool create test-zone3.rgw.buckets.data 64 64 --cluster c2 2021-03-11T00:43:53.423 INFO:teuthology.orchestra.run.smithi124.stderr:pool 'test-zone3.rgw.buckets.data' created 2021-03-11T00:43:53.443 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph osd pool application enable test-zone3.rgw.buckets.data rgw --cluster c2 2021-03-11T00:43:54.451 INFO:teuthology.orchestra.run.smithi124.stderr:enabled application 'rgw' on pool 'test-zone3.rgw.buckets.data' 2021-03-11T00:43:54.483 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'placement', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:43:54.484 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'placement', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:43:54.485 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone placement --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm modify --placement-id default-placement --compression random --cluster c2 --debug-rgw 0 --debug-ms 0 2021-03-11T00:43:54.518 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:43:54.518 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:43:54.547 INFO:teuthology.orchestra.run.smithi124.stdout:{"id":"2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05","name":"test-zone3","domain_root":"test-zone3.rgw.meta:root","control_pool":"test-zone3.rgw.control","gc_pool":"test-zone3.rgw.log:gc","lc_pool":"test-zone3.rgw.log:lc","log_pool":"test-zone3.rgw.log","intent_log_pool":"test-zone3.rgw.log:intent","usage_log_pool":"test-zone3.rgw.log:usage","roles_pool":"test-zone3.rgw.meta:roles","reshard_pool":"test-zone3.rgw.log:reshard","user_keys_pool":"test-zone3.rgw.meta:users.keys","user_email_pool":"test-zone3.rgw.meta:users.email","user_swift_pool":"test-zone3.rgw.meta:users.swift","user_uid_pool":"test-zone3.rgw.meta:users.uid","otp_pool":"test-zone3.rgw.otp","system_key":{"access_key":"8WRXV4QK6AZ20SFM","secret_key":"q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone3.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone3.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone3.rgw.buckets.non-ec","index_type":0}}],"tier_config":{"event_retention_days":7,"start_with_full_sync":false},"realm_id":""} 2021-03-11T00:43:54.548 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:43:54.549 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:43:54.549 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2021-03-11T00:43:54.630 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:43:54.631 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:44:02.629 INFO:teuthology.orchestra.run.smithi124.stderr:Sending period to new master zone 4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0 2021-03-11T00:44:03.802 INFO:teuthology.orchestra.run.smithi124.stdout:{"id":"470410ef-8c66-4339-a004-064d3e2c6c91","epoch":3,"predecessor_uuid":"b521d673-4539-4378-9527-aedf408b14d1","sync_status":[],"period_map":{"id":"470410ef-8c66-4339-a004-064d3e2c6c91","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","zones":[{"id":"2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05","name":"test-zone3","endpoints":["http://smithi124.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","name":"test-zone1","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","name":"test-zone2","endpoints":["http://smithi124.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","sync_policy":{"groups":[]}}],"short_zone_ids":[{"key":"2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05","val":906122584},{"key":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","val":3593284871},{"key":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","val":1094358664}]},"master_zonegroup":"test-zonegroup","master_zone":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","realm_name":"test-realm","realm_epoch":2} 2021-03-11T00:44:03.803 DEBUG:tasks.rgw.c2.client.1:waiting for process to exit 2021-03-11T00:44:03.815 INFO:teuthology.orchestra.run:waiting for 300 2021-03-11T00:44:03.850 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T00:44:03.845+0000 7fd9be7fc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term radosgw --rgw-frontends beast port=8001 -n client.1 --cluster c2 -k /etc/ceph/c2.client.1.keyring --log-file /var/log/ceph/rgw.c2.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.1.sock --foreground (PID: 15441) UID: 0 2021-03-11T00:44:03.850 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T00:44:03.845+0000 7fd9e1ff50c0 -1 shutting down 2021-03-11T00:44:08.430 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.1 is failed for ~0s 2021-03-11T00:44:09.918 INFO:tasks.rgw.c2.client.1:Stopped 2021-03-11T00:44:09.919 INFO:tasks.rgw.c2.client.1:Restarting daemon 2021-03-11T00:44:09.919 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8001' -n client.1 --cluster c2 -k /etc/ceph/c2.client.1.keyring --log-file /var/log/ceph/rgw.c2.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.1.sock --foreground --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c2.client.1.stdout 2>&1 2021-03-11T00:44:09.928 INFO:tasks.rgw.c2.client.1:Started 2021-03-11T00:44:09.928 DEBUG:teuthology.orchestra.run.smithi124:> curl http://smithi124.front.sepia.ceph.com:8001 2021-03-11T00:44:09.945 INFO:teuthology.orchestra.run.smithi124.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2021-03-11T00:44:09.946 INFO:teuthology.orchestra.run.smithi124.stderr: Dload Upload Total Spent Left Speed 2021-03-11T00:44:09.949 INFO:teuthology.orchestra.run.smithi124.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi124.front.sepia.ceph.com port 8001: Connection refused 2021-03-11T00:44:09.950 DEBUG:teuthology.orchestra.run:got remote process result: 7 2021-03-11T00:44:10.951 DEBUG:teuthology.orchestra.run.smithi124:> curl http://smithi124.front.sepia.ceph.com:8001 2021-03-11T00:44:10.973 INFO:teuthology.orchestra.run.smithi124.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2021-03-11T00:44:10.974 INFO:teuthology.orchestra.run.smithi124.stderr: Dload Upload Total Spent Left Speed 2021-03-11T00:44:10.974 INFO:teuthology.orchestra.run.smithi124.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 134 100 134 0 0 26800 0 --:--:-- --:--:-- --:--:-- 26800 2021-03-11T00:44:10.976 INFO:teuthology.orchestra.run.smithi124.stdout:{"Code":"MethodNotAllowed","RequestId":"tx000000000000000000001-00604967da-10e5-test-zone3","HostId":"10e5-test-zone3-test-zonegroup"} 2021-03-11T00:44:10.976 INFO:tasks.rgw_multisite:rgw multisite configuration completed 2021-03-11T00:44:10.977 INFO:teuthology.run_tasks:Running task rgw-multisite-tests... 2021-03-11T00:44:10.999 INFO:tasks.rgw_multisite_tests:cloning suite branch octopus from https://github.com/ceph/ceph.git... 2021-03-11T00:44:11.005 DEBUG:teuthology.repo_utils:Setting repo remote to https://github.com/ceph/ceph.git 2021-03-11T00:44:11.028 INFO:teuthology.repo_utils:Fetching octopus from origin 2021-03-11T00:44:12.273 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/github.com_ceph_ceph_octopus to origin/octopus 2021-03-11T00:44:13.201 INFO:tasks.rgw_multisite_tests:importing tests from /home/teuthworker/src/github.com_ceph_ceph_octopus/src/test/rgw/rgw_multi 2021-03-11T00:44:13.251 INFO:tasks.rgw_multisite_tests:creating test user.. 2021-03-11T00:44:13.252 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['user', 'create', '--uid', 'rgw-multisite-test-user', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--display-name', 'Multisite Test User', '--gen-access-key', '--gen-secret', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:44:13.255 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'user', 'create', '--uid', 'rgw-multisite-test-user', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--display-name', 'Multisite Test User', '--gen-access-key', '--gen-secret', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:44:13.255 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 user create --uid rgw-multisite-test-user --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --display-name 'Multisite Test User' --gen-access-key --gen-secret --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T00:44:13.303 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:44:13.303 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:44:13.476 INFO:teuthology.orchestra.run.smithi119.stdout:{"user_id":"rgw-multisite-test-user","display_name":"Multisite Test User","email":"","suspended":0,"max_buckets":1000,"subusers":[],"keys":[{"user":"rgw-multisite-test-user","access_key":"IXQCP5157BUHIHJ97852","secret_key":"lbUt12ldwTtJHND3v1S5HWcgXLYdUrCpqJUzLEEZ"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2021-03-11T00:44:13.502 INFO:rgw_multi.tests:meta checkpoint 2021-03-11T00:44:13.503 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:44:13.504 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:44:13.504 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:44:13.580 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:44:13.580 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:44:13.740 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=470410ef-8c66-4339-a004-064d3e2c6c91 2021-03-11T00:44:13.767 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615423453.469908_73.1","last_update":"2021-03-11T00:44:13.469908Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T00:44:13.768 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2021-03-11T00:44:13.768 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:44:13.769 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:44:13.769 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:44:13.811 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:44:13.812 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:44:13.994 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:44:13.995 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423453.469908_73.1 target= 2021-03-11T00:44:14.137 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.1 has been restored 2021-03-11T00:44:18.997 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:44:18.997 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:44:18.998 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:44:19.048 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:44:19.048 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:44:19.230 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:44:19.231 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423453.469908_73.1 target= 2021-03-11T00:44:24.234 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:44:24.234 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:44:24.235 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:44:24.276 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:44:24.277 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:44:24.469 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:44:24.470 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423453.469908_73.1 target= 2021-03-11T00:44:29.472 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:44:29.472 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:44:29.473 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:44:29.514 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:44:29.514 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:44:29.706 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:44:29.707 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423453.469908_73.1 target= 2021-03-11T00:44:34.708 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:44:34.709 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:44:34.709 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:44:34.750 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:44:34.751 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:44:34.945 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:44:34.946 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423453.469908_73.1 target= 2021-03-11T00:44:39.948 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:44:39.952 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:44:39.952 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:44:39.998 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:44:39.999 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:44:40.187 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:44:40.188 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423453.469908_73.1 target= 2021-03-11T00:44:45.189 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:44:45.191 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:44:45.192 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:44:45.233 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:44:45.233 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:44:45.414 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:44:45.414 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423453.469908_73.1 target= 2021-03-11T00:44:50.416 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:44:50.420 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:44:50.420 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:44:50.466 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:44:50.466 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:44:50.640 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:44:50.640 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423453.469908_73.1 target= 2021-03-11T00:44:55.641 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:44:55.642 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:44:55.643 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:44:55.691 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:44:55.692 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:44:55.897 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:44:55.897 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423453.469908_73.1 target= 2021-03-11T00:45:00.899 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:45:00.900 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:45:00.900 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:45:00.941 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:45:00.941 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:45:01.136 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:45:01.136 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423453.469908_73.1 target= 2021-03-11T00:45:06.138 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:45:06.142 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:45:06.143 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:45:06.184 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:45:06.185 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:45:06.379 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:45:06.380 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423453.469908_73.1 target= 2021-03-11T00:45:11.382 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:45:11.384 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:45:11.384 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:45:11.425 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:45:11.425 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:45:11.625 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:45:11.625 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423453.469908_73.1 target= 2021-03-11T00:45:16.626 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:45:16.627 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:45:16.627 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:45:16.666 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:45:16.667 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:45:16.861 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:45:16.862 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423453.469908_73.1 target= 2021-03-11T00:45:21.863 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:45:21.863 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:45:21.864 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:45:21.908 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:45:21.908 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:45:22.092 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:45:22.092 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423453.469908_73.1 target= 2021-03-11T00:45:27.094 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:45:27.095 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:45:27.095 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:45:27.136 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:45:27.136 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:45:27.327 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:45:27.327 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423453.469908_73.1 target= 2021-03-11T00:45:32.328 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:45:32.329 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:45:32.329 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:45:32.370 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:45:32.370 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:45:32.552 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:45:32.553 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423453.469908_73.1 target= 2021-03-11T00:45:37.554 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:45:37.556 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:45:37.556 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:45:37.598 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:45:37.598 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:45:37.771 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:45:37.772 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423453.469908_73.1 target= 2021-03-11T00:45:42.773 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:45:42.775 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:45:42.776 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:45:42.818 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:45:42.819 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:45:43.009 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:45:43.010 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423453.469908_73.1 target= 2021-03-11T00:45:48.011 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:45:48.026 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:45:48.027 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:45:48.067 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:45:48.067 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:45:48.276 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:45:48.277 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423453.469908_73.1 target= 2021-03-11T00:45:53.278 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:45:53.462 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:45:53.463 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:45:53.507 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:45:53.508 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:45:53.696 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:45:53.697 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423453.469908_73.1 target= 2021-03-11T00:45:58.698 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:45:58.700 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:45:58.701 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:45:58.742 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:45:58.743 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:45:58.933 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:45:58.934 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423453.469908_73.1 target= 2021-03-11T00:46:03.936 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:03.938 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:03.938 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:03.979 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:03.980 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:04.160 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:46:04.161 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423453.469908_73.1 target= 2021-03-11T00:46:09.162 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:09.164 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:09.165 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:09.205 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:09.205 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:09.399 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423453.469908_73.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:44:13.469908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:46:09.403 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2021-03-11T00:46:09.405 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T00:46:09.406 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:09.406 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:09.422 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:09.489 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:09.489 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:09.740 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423453.469908_73.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:44:13.469908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:46:09.740 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T00:46:09.741 INFO:rgw_multi.tests:meta checkpoint 2021-03-11T00:46:09.741 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:09.741 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:09.742 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:09.785 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:09.786 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:09.943 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=470410ef-8c66-4339-a004-064d3e2c6c91 2021-03-11T00:46:09.973 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615423453.469908_73.1","last_update":"2021-03-11T00:44:13.469908Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T00:46:09.974 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2021-03-11T00:46:09.974 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:09.975 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:09.975 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:10.014 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:10.015 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:10.180 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423453.469908_73.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:44:13.469908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:46:10.181 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2021-03-11T00:46:10.181 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T00:46:10.182 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:10.183 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:10.183 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:10.262 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:10.262 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:10.520 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423453.469908_73.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:44:13.469908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:46:10.521 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T00:46:10.521 INFO:tasks.rgw_multisite_tests:running rgw multisite tests on '/home/teuthworker/src/github.com_ceph_ceph_octopus/src/test/rgw/rgw_multi' with args=['tests.py', 'tests_ps.py'] 2021-03-11T00:46:10.589 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=nolznk-1 2021-03-11T00:46:13.795 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=nolznk-2 2021-03-11T00:46:17.506 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:17.507 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:17.508 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:17.550 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:17.551 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:17.699 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=470410ef-8c66-4339-a004-064d3e2c6c91 2021-03-11T00:46:17.737 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615423453.469908_73.1","last_update":"2021-03-11T00:44:13.469908Z"},{"marker":"1_1615423573.832639_40.1","last_update":"2021-03-11T00:46:13.832639Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T00:46:17.738 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2021-03-11T00:46:17.739 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:17.739 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:17.740 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:17.781 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:17.781 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:17.969 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423453.469908_73.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:44:13.469908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423573.832639_40.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:13.832639Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:46:17.969 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2021-03-11T00:46:17.970 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T00:46:17.971 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:17.971 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:17.972 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:18.053 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:18.054 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:18.306 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423453.469908_73.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:44:13.469908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423573.779224_33.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:13.779224Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:46:18.306 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=1 master=1_1615423573.832639_40.1 target=1_1615423573.779224_33.1 2021-03-11T00:46:23.308 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:23.309 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:23.310 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:23.351 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:23.351 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:23.608 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423453.469908_73.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:44:13.469908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423573.832639_40.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:13.832639Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:46:23.608 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T00:46:23.628 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_create ... ok 2021-03-11T00:46:23.632 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=nolznk-3 2021-03-11T00:46:23.659 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=nolznk-4 2021-03-11T00:46:23.712 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:23.713 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:23.714 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:23.751 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:23.752 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:23.926 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=470410ef-8c66-4339-a004-064d3e2c6c91 2021-03-11T00:46:23.960 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615423453.469908_73.1","last_update":"2021-03-11T00:44:13.469908Z"},{"marker":"1_1615423583.694980_48.1","last_update":"2021-03-11T00:46:23.694980Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T00:46:23.961 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2021-03-11T00:46:23.961 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:23.962 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:23.963 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:24.002 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:24.002 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:24.193 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423453.469908_73.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:44:13.469908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423573.832639_40.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:13.832639Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:46:24.193 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=1 master=1_1615423583.694980_48.1 target=1_1615423573.832639_40.1 2021-03-11T00:46:29.194 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:29.195 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:29.196 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:29.241 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:29.241 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:29.442 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423453.469908_73.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:44:13.469908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423583.694980_48.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:23.694980Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:46:29.442 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2021-03-11T00:46:29.443 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T00:46:29.443 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:29.444 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:29.445 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:29.522 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:29.522 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:29.777 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423453.469908_73.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:44:13.469908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423583.694980_48.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:23.694980Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:46:29.777 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T00:46:30.078 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:30.079 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:30.080 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:30.118 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:30.119 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:30.283 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=470410ef-8c66-4339-a004-064d3e2c6c91 2021-03-11T00:46:30.316 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615423453.469908_73.1","last_update":"2021-03-11T00:44:13.469908Z"},{"marker":"1_1615423589.969156_67.1","last_update":"2021-03-11T00:46:29.969156Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T00:46:30.317 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2021-03-11T00:46:30.318 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:30.318 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:30.319 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:30.360 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:30.360 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:30.539 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423453.469908_73.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:44:13.469908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423583.694980_48.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:23.694980Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:46:30.539 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=1 master=1_1615423589.969156_67.1 target=1_1615423583.694980_48.1 2021-03-11T00:46:35.541 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:35.546 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:35.547 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:35.587 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:35.588 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:35.785 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423453.469908_73.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:44:13.469908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423589.969156_67.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:29.969156Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:46:35.786 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2021-03-11T00:46:35.786 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T00:46:35.787 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:35.787 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:35.788 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:35.867 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:35.868 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:36.126 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423453.469908_73.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:44:13.469908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423589.969156_67.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:29.969156Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:46:36.127 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T00:46:36.144 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_recreate ... ok 2021-03-11T00:46:36.145 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=nolznk-5 2021-03-11T00:46:36.171 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=nolznk-6 2021-03-11T00:46:36.220 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:36.220 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:36.221 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:36.258 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:36.259 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:36.423 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=470410ef-8c66-4339-a004-064d3e2c6c91 2021-03-11T00:46:36.453 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615423453.469908_73.1","last_update":"2021-03-11T00:44:13.469908Z"},{"marker":"1_1615423596.201627_78.1","last_update":"2021-03-11T00:46:36.201627Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T00:46:36.454 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2021-03-11T00:46:36.454 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:36.455 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:36.456 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:36.499 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:36.500 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:36.673 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423453.469908_73.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:44:13.469908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423589.969156_67.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:29.969156Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:46:36.674 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=1 master=1_1615423596.201627_78.1 target=1_1615423589.969156_67.1 2021-03-11T00:46:41.675 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:41.687 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:41.687 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:41.728 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:41.729 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:41.917 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423453.469908_73.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:44:13.469908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423596.201627_78.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:36.201627Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:46:41.918 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2021-03-11T00:46:41.919 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T00:46:41.919 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:41.919 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:41.920 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:42.001 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:42.001 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:42.262 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423453.469908_73.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:44:13.469908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423596.201627_78.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:36.201627Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:46:42.263 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T00:46:42.388 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:42.389 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:42.390 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:42.428 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:42.428 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:42.584 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=470410ef-8c66-4339-a004-064d3e2c6c91 2021-03-11T00:46:42.618 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615423453.469908_73.1","last_update":"2021-03-11T00:44:13.469908Z"},{"marker":"1_1615423602.361914_82.1","last_update":"2021-03-11T00:46:42.361914Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T00:46:42.618 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2021-03-11T00:46:42.619 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:42.619 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:42.620 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:42.660 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:42.661 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:42.853 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423453.469908_73.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:44:13.469908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423596.201627_78.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:36.201627Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:46:42.854 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=1 master=1_1615423602.361914_82.1 target=1_1615423596.201627_78.1 2021-03-11T00:46:47.856 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:47.857 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:47.858 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:47.906 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:47.906 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:48.107 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423453.469908_73.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:44:13.469908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423602.361914_82.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:42.361914Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:46:48.108 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2021-03-11T00:46:48.109 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T00:46:48.110 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:48.111 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:48.113 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:48.206 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:48.207 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:48.462 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423453.469908_73.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:44:13.469908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423602.361914_82.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:42.361914Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:46:48.462 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T00:46:48.477 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_remove ... ok 2021-03-11T00:46:48.480 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=nolznk-7 2021-03-11T00:46:48.509 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=nolznk-8 2021-03-11T00:46:48.959 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:48.960 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:48.960 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:48.999 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:49.000 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:49.159 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=470410ef-8c66-4339-a004-064d3e2c6c91 2021-03-11T00:46:49.189 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615423453.469908_73.1","last_update":"2021-03-11T00:44:13.469908Z"},{"marker":"1_1615423608.538319_90.1","last_update":"2021-03-11T00:46:48.538319Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T00:46:49.189 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2021-03-11T00:46:49.190 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:49.190 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:49.191 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:49.232 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:49.233 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:49.422 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423453.469908_73.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:44:13.469908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423602.361914_82.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:42.361914Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:46:49.423 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=1 master=1_1615423608.538319_90.1 target=1_1615423602.361914_82.1 2021-03-11T00:46:49.684 INFO:tasks.rgw.c2.client.0.smithi124.stdout:2021-03-11T00:46:49.681+0000 7fd2b3ff7700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T00:46:49.684 INFO:tasks.rgw.c2.client.0.smithi124.stdout:2021-03-11T00:46:49.681+0000 7fd2b67fc700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T00:46:49.685 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T00:46:49.681+0000 7fc63e7ec700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T00:46:49.685 INFO:tasks.rgw.c2.client.0.smithi124.stdout:2021-03-11T00:46:49.681+0000 7fd2b5ffb700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T00:46:49.686 INFO:tasks.rgw.c2.client.0.smithi124.stdout:2021-03-11T00:46:49.681+0000 7fd2b77fe700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T00:46:49.687 INFO:tasks.rgw.c2.client.0.smithi124.stdout:2021-03-11T00:46:49.681+0000 7fd2aefed700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T00:46:49.687 INFO:tasks.rgw.c2.client.0.smithi124.stdout:2021-03-11T00:46:49.681+0000 7fd2b37f6700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T00:46:49.688 INFO:tasks.rgw.c2.client.0.smithi124.stdout:2021-03-11T00:46:49.681+0000 7fd2b37f6700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T00:46:49.689 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T00:46:49.681+0000 7fc643ff7700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T00:46:49.689 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T00:46:49.681+0000 7fc646ffd700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T00:46:49.689 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T00:46:49.681+0000 7fc65e7fc700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T00:46:49.690 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T00:46:49.681+0000 7fc65e7fc700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T00:46:49.690 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T00:46:49.685+0000 7fc63dfeb700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T00:46:49.691 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T00:46:49.685+0000 7fc63b7e6700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T00:46:54.424 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:54.432 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:54.433 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:54.474 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:54.475 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:54.653 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423453.469908_73.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:44:13.469908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423608.538319_90.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:48.538319Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:46:54.653 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2021-03-11T00:46:54.654 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T00:46:54.654 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:54.655 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:54.656 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:54.733 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:54.734 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:54.988 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423453.469908_73.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:44:13.469908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423608.538319_90.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:48.538319Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:46:54.989 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T00:46:54.989 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-7', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:54.990 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-7', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:54.990 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-7 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:55.030 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:55.030 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:55.213 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":"00000000001.6.5"},{"key":2,"val":""},{"key":3,"val":"00000000001.14.5"},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":"00000000001.4.5"},{"key":8,"val":"00000000001.11.5"},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:46:55.214 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-7 2021-03-11T00:46:55.214 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-7', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:55.215 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-7', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:55.215 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-7 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:55.259 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:55.260 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:55.459 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"myobj","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.6.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"_myobj","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.14.5"}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"&","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.4.5"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":":","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.11.5"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:46:55.460 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-7 2021-03-11T00:46:55.461 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-7 zones={test-zone2, test-zone1} 2021-03-11T00:46:55.484 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:46:55.535 DEBUG:tasks.rgw_multi.zone_rados:o=& 2021-03-11T00:46:55.536 DEBUG:tasks.rgw_multi.zone_rados:o=: 2021-03-11T00:46:55.536 DEBUG:tasks.rgw_multi.zone_rados:o=_myobj 2021-03-11T00:46:55.537 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2021-03-11T00:46:55.537 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:46:55.543 DEBUG:tasks.rgw_multi.zone_rados:o=& 2021-03-11T00:46:55.544 DEBUG:tasks.rgw_multi.zone_rados:o=: 2021-03-11T00:46:55.544 DEBUG:tasks.rgw_multi.zone_rados:o=_myobj 2021-03-11T00:46:55.544 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2021-03-11T00:46:55.601 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2021-03-11T00:46:55.654 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2021-03-11T00:46:55.761 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2021-03-11T00:46:55.864 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2021-03-11T00:46:55.919 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2021-03-11T00:46:56.026 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2021-03-11T00:46:56.127 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2021-03-11T00:46:56.183 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2021-03-11T00:46:56.301 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2021-03-11T00:46:56.404 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2021-03-11T00:46:56.464 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2021-03-11T00:46:56.573 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2021-03-11T00:46:56.684 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-7 zones={test-zone2, test-zone1} 2021-03-11T00:46:56.685 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-7', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:56.687 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-7', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:56.687 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-7 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:56.729 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:56.730 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:56.927 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":"00000000001.6.5"},{"key":2,"val":""},{"key":3,"val":"00000000001.14.5"},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":"00000000001.4.5"},{"key":8,"val":"00000000001.11.5"},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:46:56.927 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-7 2021-03-11T00:46:56.928 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-7', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:56.929 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-7', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:56.929 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-7 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:56.972 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:56.973 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:57.228 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.6.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.14.5"}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.4.5"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.11.5"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:46:57.229 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-7 2021-03-11T00:46:57.229 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-8', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:57.230 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-8', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:57.231 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-8 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:57.312 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:57.313 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:57.499 INFO:teuthology.orchestra.run.smithi124.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":"00000000001.7.5"},{"key":2,"val":""},{"key":3,"val":"00000000001.4.5"},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":"00000000001.2.5"},{"key":8,"val":"00000000001.4.5"},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:46:57.500 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-8 2021-03-11T00:46:57.500 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-8', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:57.501 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-8', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:57.502 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-8 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:57.545 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:57.546 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:57.758 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"myobj","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.7.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"_myobj","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.4.5"}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"&","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.2.5"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":":","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.4.5"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:46:57.758 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-8 2021-03-11T00:46:57.759 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-8 zones={test-zone1, test-zone2} 2021-03-11T00:46:57.773 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:46:57.827 DEBUG:tasks.rgw_multi.zone_rados:o=& 2021-03-11T00:46:57.828 DEBUG:tasks.rgw_multi.zone_rados:o=: 2021-03-11T00:46:57.828 DEBUG:tasks.rgw_multi.zone_rados:o=_myobj 2021-03-11T00:46:57.829 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2021-03-11T00:46:57.829 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:46:57.835 DEBUG:tasks.rgw_multi.zone_rados:o=& 2021-03-11T00:46:57.836 DEBUG:tasks.rgw_multi.zone_rados:o=: 2021-03-11T00:46:57.837 DEBUG:tasks.rgw_multi.zone_rados:o=_myobj 2021-03-11T00:46:57.837 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2021-03-11T00:46:57.891 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2021-03-11T00:46:57.945 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2021-03-11T00:46:58.053 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2021-03-11T00:46:58.156 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2021-03-11T00:46:58.212 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2021-03-11T00:46:58.322 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2021-03-11T00:46:58.425 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2021-03-11T00:46:58.478 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2021-03-11T00:46:58.586 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2021-03-11T00:46:58.690 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2021-03-11T00:46:58.743 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2021-03-11T00:46:58.855 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2021-03-11T00:46:58.993 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-8 zones={test-zone1, test-zone2} 2021-03-11T00:46:58.994 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_object_sync ... ok 2021-03-11T00:46:58.998 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=nolznk-9 2021-03-11T00:46:59.031 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=nolznk-10 2021-03-11T00:46:59.209 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:59.213 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:59.214 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:59.258 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:59.258 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:59.427 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=470410ef-8c66-4339-a004-064d3e2c6c91 2021-03-11T00:46:59.457 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615423619.066252_94.1","last_update":"2021-03-11T00:46:59.066252Z"},{"marker":"1_1615423619.033309_101.1","last_update":"2021-03-11T00:46:59.033309Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T00:46:59.457 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2021-03-11T00:46:59.458 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:59.458 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:46:59.459 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:46:59.499 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:46:59.500 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:46:59.681 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423453.469908_73.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:44:13.469908Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423608.538319_90.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:48.538319Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:46:59.682 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423619.066252_94.1 target=1_1615423453.469908_73.1, shard=1 master=1_1615423619.033309_101.1 target=1_1615423608.538319_90.1 2021-03-11T00:47:04.684 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:04.692 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:04.693 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:04.737 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:04.738 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:04.931 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423619.066252_94.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:46:59.066252Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:47:04.931 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2021-03-11T00:47:04.932 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T00:47:04.932 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:04.933 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:04.933 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:05.018 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:05.019 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:05.273 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423619.066252_94.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:46:59.066252Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:47:05.273 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T00:47:05.274 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-9', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:05.275 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-9', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:05.275 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-9 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:05.317 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:05.318 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:05.527 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":"00000000001.9.5"},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:47:05.527 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-9 2021-03-11T00:47:05.528 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-9', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:05.528 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-9', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:05.529 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-9 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:05.573 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:05.573 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:05.789 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"myobj","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.9.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:47:05.790 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-9 2021-03-11T00:47:05.791 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-9 zones={test-zone2, test-zone1} 2021-03-11T00:47:05.812 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:47:05.863 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2021-03-11T00:47:05.863 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:47:05.869 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2021-03-11T00:47:05.924 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2021-03-11T00:47:05.982 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2021-03-11T00:47:06.090 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2021-03-11T00:47:06.192 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-9 zones={test-zone2, test-zone1} 2021-03-11T00:47:06.193 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-9', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:06.193 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-9', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:06.194 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-9 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:06.233 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:06.233 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:06.440 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":"00000000001.9.5"},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:47:06.440 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-9 2021-03-11T00:47:06.441 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-9', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:06.442 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-9', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:06.442 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-9 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:06.489 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:06.490 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:06.739 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.9.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:47:06.740 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-9 2021-03-11T00:47:06.740 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-10', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:06.741 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-10', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:06.741 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-10 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:06.822 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:06.822 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:06.996 INFO:teuthology.orchestra.run.smithi124.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":"00000000001.10.5"},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:47:06.997 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-10 2021-03-11T00:47:06.997 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-10', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:06.998 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-10', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:06.998 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-10 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:07.043 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:07.043 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:07.258 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"myobj","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.10.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:47:07.258 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-10 2021-03-11T00:47:07.259 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-10 zones={test-zone1, test-zone2} 2021-03-11T00:47:07.268 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:47:07.315 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2021-03-11T00:47:07.315 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:47:07.321 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2021-03-11T00:47:07.380 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2021-03-11T00:47:07.435 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2021-03-11T00:47:07.545 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2021-03-11T00:47:07.648 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-10 zones={test-zone1, test-zone2} 2021-03-11T00:47:07.665 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-9', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:07.666 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-9', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:07.666 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-9 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:07.704 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:07.705 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:07.888 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":"00000000002.11.5"},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:47:07.889 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-9 2021-03-11T00:47:07.889 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-9', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:07.890 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-9', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:07.891 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-9 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:07.937 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:07.937 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:08.124 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"myobj","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.9.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:47:08.125 WARNING:rgw_multi.tests:bucket nolznk-9 zone test-zone2 behind zone test-zone1: shard=1 master=00000000002.11.5 target=00000000001.9.5 2021-03-11T00:47:13.127 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-9', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:13.128 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-9', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:13.129 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-9 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:13.173 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:13.174 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:13.370 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"myobj","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000002.11.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:47:13.371 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-9 2021-03-11T00:47:13.373 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-9 zones={test-zone2, test-zone1} 2021-03-11T00:47:13.382 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:47:13.431 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:47:13.488 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-9 zones={test-zone2, test-zone1} 2021-03-11T00:47:13.489 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-9', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:13.490 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-9', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:13.491 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-9 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:13.532 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:13.533 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:13.711 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":"00000000002.11.5"},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:47:13.711 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-9 2021-03-11T00:47:13.712 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-9', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:13.712 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-9', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:13.713 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-9 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:13.757 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:13.757 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:14.013 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.11.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:47:14.014 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-9 2021-03-11T00:47:14.030 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-10', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:14.030 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-10', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:14.031 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-10 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:14.098 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:14.099 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:14.281 INFO:teuthology.orchestra.run.smithi124.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":"00000000002.13.5"},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:47:14.281 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-10 2021-03-11T00:47:14.282 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-10', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:14.282 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-10', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:14.283 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-10 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:14.329 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:14.329 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:14.544 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"myobj","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.10.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:47:14.544 WARNING:rgw_multi.tests:bucket nolznk-10 zone test-zone1 behind zone test-zone2: shard=1 master=00000000002.13.5 target=00000000001.10.5 2021-03-11T00:47:19.549 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-10', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:19.553 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-10', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:19.554 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-10 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:19.607 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:19.608 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:19.816 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"myobj","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000002.13.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:47:19.818 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-10 2021-03-11T00:47:19.821 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-10 zones={test-zone1, test-zone2} 2021-03-11T00:47:19.845 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:47:19.918 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:47:19.994 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-10 zones={test-zone1, test-zone2} 2021-03-11T00:47:19.995 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_object_delete ... ok 2021-03-11T00:47:19.999 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=nolznk-11 2021-03-11T00:47:20.031 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=nolznk-12 2021-03-11T00:47:20.117 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:20.119 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:20.120 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:20.161 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:20.162 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:20.328 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=470410ef-8c66-4339-a004-064d3e2c6c91 2021-03-11T00:47:20.360 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615423640.114087_114.1","last_update":"2021-03-11T00:47:20.114087Z"},{"marker":"1_1615423619.033309_101.1","last_update":"2021-03-11T00:46:59.033309Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T00:47:20.360 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2021-03-11T00:47:20.361 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:20.362 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:20.363 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:20.406 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:20.407 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:20.577 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423640.106848_113.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:47:20.106848Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:47:20.578 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423640.114087_114.1 target=1_1615423640.106848_113.1 2021-03-11T00:47:25.581 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:25.589 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:25.590 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:25.630 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:25.630 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:25.813 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423640.114087_114.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:47:20.114087Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:47:25.813 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2021-03-11T00:47:25.814 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T00:47:25.814 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:25.815 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:25.815 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:25.896 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:25.896 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:26.151 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423640.114087_114.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:47:20.114087Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:47:26.151 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T00:47:26.203 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-11', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:26.203 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-11', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:26.204 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-11 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:26.245 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:26.246 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:26.427 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":"00000000002.10.10"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:47:26.427 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-11 2021-03-11T00:47:26.428 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:26.428 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:26.429 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:26.476 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:26.476 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:26.662 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:47:26.662 WARNING:rgw_multi.tests:bucket nolznk-11 zone test-zone2 behind zone test-zone1: shard=5 master=00000000002.10.10 target= 2021-03-11T00:47:31.664 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:31.795 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:31.796 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:31.840 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:31.840 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:32.031 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"KILWhg.oBlHSDRCHQtSEgjWJFku2kfZ","ns":""},"count":1},"inc_marker":{"position":"00000000003.12.9"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:47:32.035 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-11 2021-03-11T00:47:32.035 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-11', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:32.036 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-11', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:32.037 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-11 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:32.077 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:32.077 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:32.269 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":"00000000003.12.9"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:47:32.270 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-11 2021-03-11T00:47:32.270 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:32.271 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:32.272 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:32.315 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:32.315 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:32.567 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000003.12.9"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:47:32.568 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-11 2021-03-11T00:47:32.619 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-12', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:32.619 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-12', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:32.620 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-12 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:32.660 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:32.660 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:32.830 INFO:teuthology.orchestra.run.smithi124.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":"00000000002.9.10"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:47:32.831 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-12 2021-03-11T00:47:32.831 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-12', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:32.832 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-12', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:32.833 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-12 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:32.877 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:32.878 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:33.086 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:47:33.087 WARNING:rgw_multi.tests:bucket nolznk-12 zone test-zone1 behind zone test-zone2: shard=5 master=00000000002.9.10 target= 2021-03-11T00:47:38.089 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-12', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:38.090 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-12', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:38.091 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-12 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:38.147 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:38.147 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:38.353 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"gS3h4uWLGJ2yhaPxOtwlzrFKvvrbWZ-","ns":""},"count":1},"inc_marker":{"position":"00000000003.11.9"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:47:38.354 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-12 2021-03-11T00:47:39.186 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-11', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:39.187 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-11', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:39.187 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-11 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:39.228 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:39.229 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:39.421 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":"00000000010.30.5"},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":"00000000003.12.9"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:47:39.421 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-11 2021-03-11T00:47:39.422 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:39.423 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:39.423 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:39.467 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:39.467 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:39.647 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"full-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000010.30.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"KILWhg.oBlHSDRCHQtSEgjWJFku2kfZ","ns":""},"count":1},"inc_marker":{"position":"00000000003.12.9"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:47:39.648 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-11 2021-03-11T00:47:39.649 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-11', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:39.649 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-11', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:39.650 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-11 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:39.693 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:39.694 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:39.906 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":"00000000010.30.5"},{"key":2,"val":""},{"key":3,"val":"00000000002.20.10"},{"key":4,"val":""},{"key":5,"val":"00000000003.12.9"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:47:39.907 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-11 2021-03-11T00:47:39.907 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:39.908 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:39.909 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:39.955 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:39.956 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:40.210 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000010.30.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000003.12.9"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:47:40.211 WARNING:rgw_multi.tests:bucket nolznk-11 zone test-zone3 behind zone test-zone1: shard=3 master=00000000002.20.10 target= 2021-03-11T00:47:45.212 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:45.222 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:45.222 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:45.269 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:45.269 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:45.524 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000010.30.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.20.10"}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000003.12.9"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:47:45.525 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-11 2021-03-11T00:47:45.525 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:45.526 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:45.526 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:45.605 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:45.606 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:45.782 INFO:teuthology.orchestra.run.smithi124.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":"00000000002.17.10"},{"key":2,"val":""},{"key":3,"val":"00000000010.41.5"},{"key":4,"val":""},{"key":5,"val":"00000000002.10.10"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:47:45.783 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-11 2021-03-11T00:47:45.784 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-11', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:45.785 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-11', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:45.786 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-11 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:45.832 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:45.832 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:46.045 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj-test-zone1","instance":"McIuVZRR236hBHDQ3NV4fXLT6v95t6U","ns":""},"count":1},"inc_marker":{"position":"00000000002.17.10"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj-test-zone2","instance":"da5dd1-LCFaD76mX1CMkvIdwaUSljoC","ns":""},"count":1},"inc_marker":{"position":"00000000010.41.5"}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"KILWhg.oBlHSDRCHQtSEgjWJFku2kfZ","ns":""},"count":1},"inc_marker":{"position":"00000000002.10.10"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:47:46.046 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-11 2021-03-11T00:47:46.047 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-11 zones={test-zone2, test-zone1} 2021-03-11T00:47:46.075 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:47:46.082 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2021-03-11T00:47:46.083 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2021-03-11T00:47:46.084 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2021-03-11T00:47:46.084 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:47:46.090 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2021-03-11T00:47:46.090 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2021-03-11T00:47:46.091 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2021-03-11T00:47:46.101 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2021-03-11T00:47:46.111 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2021-03-11T00:47:46.176 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2021-03-11T00:47:46.236 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2021-03-11T00:47:46.292 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2021-03-11T00:47:46.409 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2021-03-11T00:47:46.508 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2021-03-11T00:47:46.562 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2021-03-11T00:47:46.675 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2021-03-11T00:47:46.778 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-11 zones={test-zone2, test-zone1} 2021-03-11T00:47:46.779 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-12', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:46.780 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-12', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:46.781 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-12 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:46.821 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:46.821 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:47.029 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":"00000000010.23.5"},{"key":2,"val":""},{"key":3,"val":"00000000002.12.10"},{"key":4,"val":""},{"key":5,"val":"00000000002.6.10"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:47:47.029 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-12 2021-03-11T00:47:47.030 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-12', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:47.030 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-12', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:47.031 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-12 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:47.075 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:47.075 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:47.259 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj-test-zone1","instance":"95K-tIYWIOr8Q8PhyztL3fZpDaVRQ3N","ns":""},"count":1},"inc_marker":{"position":"00000000010.23.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj-test-zone2","instance":"IoGVAznPywpWoDPeunnFo6cMKKGv4-A","ns":""},"count":1},"inc_marker":{"position":"00000000002.12.10"}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"gS3h4uWLGJ2yhaPxOtwlzrFKvvrbWZ-","ns":""},"count":1},"inc_marker":{"position":"00000000002.6.10"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:47:47.259 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-12 2021-03-11T00:47:47.260 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-12', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:47.260 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-12', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:47.261 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-12 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:47.301 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:47.302 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:47.511 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":"00000000010.23.5"},{"key":2,"val":""},{"key":3,"val":"00000000002.12.10"},{"key":4,"val":""},{"key":5,"val":"00000000002.6.10"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:47:47.512 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-12 2021-03-11T00:47:47.513 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-12', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:47.513 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-12', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:47.514 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-12 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:47.554 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:47.555 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:47.809 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000010.23.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.12.10"}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.6.10"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:47:47.810 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-12 2021-03-11T00:47:47.811 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-12', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:47.811 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-12', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:47.812 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-12 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:47.891 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:47.892 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:48.065 INFO:teuthology.orchestra.run.smithi124.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":"00000000002.9.10"},{"key":2,"val":""},{"key":3,"val":"00000000010.37.5"},{"key":4,"val":""},{"key":5,"val":"00000000003.11.9"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:47:48.066 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-12 2021-03-11T00:47:48.067 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-12', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:48.068 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-12', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:48.069 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-12 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:48.116 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:48.117 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:48.324 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj-test-zone1","instance":"95K-tIYWIOr8Q8PhyztL3fZpDaVRQ3N","ns":""},"count":1},"inc_marker":{"position":"00000000002.9.10"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj-test-zone2","instance":"IoGVAznPywpWoDPeunnFo6cMKKGv4-A","ns":""},"count":1},"inc_marker":{"position":"00000000010.37.5"}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"gS3h4uWLGJ2yhaPxOtwlzrFKvvrbWZ-","ns":""},"count":1},"inc_marker":{"position":"00000000003.11.9"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:47:48.325 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-12 2021-03-11T00:47:48.326 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-12 zones={test-zone2, test-zone1} 2021-03-11T00:47:48.338 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:47:48.387 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2021-03-11T00:47:48.387 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2021-03-11T00:47:48.388 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2021-03-11T00:47:48.388 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:47:48.396 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2021-03-11T00:47:48.397 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2021-03-11T00:47:48.398 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2021-03-11T00:47:48.453 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2021-03-11T00:47:48.467 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2021-03-11T00:47:48.538 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2021-03-11T00:47:48.605 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2021-03-11T00:47:48.662 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2021-03-11T00:47:48.769 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2021-03-11T00:47:48.872 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2021-03-11T00:47:48.927 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2021-03-11T00:47:49.044 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2021-03-11T00:47:49.149 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-12 zones={test-zone2, test-zone1} 2021-03-11T00:47:49.423 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-11', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:49.424 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-11', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:49.425 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-11 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:49.474 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:49.884 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:49.886 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":"00000000011.34.5"},{"key":2,"val":""},{"key":3,"val":"00000000002.20.10"},{"key":4,"val":""},{"key":5,"val":"00000000003.12.9"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:47:49.887 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-11 2021-03-11T00:47:49.887 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:49.888 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:49.888 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:49.934 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:49.934 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:50.113 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj-test-zone1","instance":"McIuVZRR236hBHDQ3NV4fXLT6v95t6U","ns":""},"count":1},"inc_marker":{"position":"00000000010.30.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj-test-zone2","instance":"da5dd1-LCFaD76mX1CMkvIdwaUSljoC","ns":""},"count":1},"inc_marker":{"position":"00000000002.20.10"}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"KILWhg.oBlHSDRCHQtSEgjWJFku2kfZ","ns":""},"count":1},"inc_marker":{"position":"00000000003.12.9"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:47:50.114 WARNING:rgw_multi.tests:bucket nolznk-11 zone test-zone2 behind zone test-zone1: shard=1 master=00000000011.34.5 target=00000000010.30.5 2021-03-11T00:47:55.116 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:55.117 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:55.117 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:55.167 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:55.168 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:55.353 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj-test-zone1","instance":"McIuVZRR236hBHDQ3NV4fXLT6v95t6U","ns":""},"count":1},"inc_marker":{"position":"00000000011.34.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj-test-zone2","instance":"da5dd1-LCFaD76mX1CMkvIdwaUSljoC","ns":""},"count":1},"inc_marker":{"position":"00000000003.23.5"}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"KILWhg.oBlHSDRCHQtSEgjWJFku2kfZ","ns":""},"count":1},"inc_marker":{"position":"00000000003.12.9"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:47:55.353 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-11 2021-03-11T00:47:55.354 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-11', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:55.355 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-11', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:55.355 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-11 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:55.400 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:55.401 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:55.619 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":"00000000011.34.5"},{"key":2,"val":""},{"key":3,"val":"00000000003.23.5"},{"key":4,"val":""},{"key":5,"val":"00000000003.12.9"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:47:55.620 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-11 2021-03-11T00:47:55.621 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:55.623 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:55.624 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:55.668 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:55.669 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:55.927 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000011.34.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000003.23.5"}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000003.12.9"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:47:55.928 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-11 2021-03-11T00:47:55.929 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:55.930 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:55.931 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:56.011 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:56.011 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:56.184 INFO:teuthology.orchestra.run.smithi124.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":"00000000003.20.5"},{"key":2,"val":""},{"key":3,"val":"00000000011.45.5"},{"key":4,"val":""},{"key":5,"val":"00000000002.10.10"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:47:56.184 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-11 2021-03-11T00:47:56.185 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-11', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:56.185 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-11', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:56.186 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-11 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:56.231 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:56.232 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:56.452 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj-test-zone1","instance":"McIuVZRR236hBHDQ3NV4fXLT6v95t6U","ns":""},"count":1},"inc_marker":{"position":"00000000003.20.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj-test-zone2","instance":"da5dd1-LCFaD76mX1CMkvIdwaUSljoC","ns":""},"count":1},"inc_marker":{"position":"00000000011.45.5"}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"KILWhg.oBlHSDRCHQtSEgjWJFku2kfZ","ns":""},"count":1},"inc_marker":{"position":"00000000002.10.10"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:47:56.453 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-11 2021-03-11T00:47:56.454 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-11 zones={test-zone2, test-zone1} 2021-03-11T00:47:56.466 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:47:56.476 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2021-03-11T00:47:56.476 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2021-03-11T00:47:56.477 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2021-03-11T00:47:56.478 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:47:56.531 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2021-03-11T00:47:56.532 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2021-03-11T00:47:56.533 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2021-03-11T00:47:56.596 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2021-03-11T00:47:56.626 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2021-03-11T00:47:56.726 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2021-03-11T00:47:56.806 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2021-03-11T00:47:56.885 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2021-03-11T00:47:57.065 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2021-03-11T00:47:57.228 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2021-03-11T00:47:57.298 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2021-03-11T00:47:57.462 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2021-03-11T00:47:57.602 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-11 zones={test-zone2, test-zone1} 2021-03-11T00:47:57.609 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-12', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:57.611 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-12', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:57.615 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-12 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:57.659 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:57.659 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:57.844 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":"00000000011.27.5"},{"key":2,"val":""},{"key":3,"val":"00000000003.15.5"},{"key":4,"val":""},{"key":5,"val":"00000000002.6.10"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:47:57.851 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-12 2021-03-11T00:47:57.851 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-12', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:57.853 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-12', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:57.854 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-12 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:57.907 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:57.908 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:58.096 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj-test-zone1","instance":"95K-tIYWIOr8Q8PhyztL3fZpDaVRQ3N","ns":""},"count":1},"inc_marker":{"position":"00000000011.27.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj-test-zone2","instance":"IoGVAznPywpWoDPeunnFo6cMKKGv4-A","ns":""},"count":1},"inc_marker":{"position":"00000000003.15.5"}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"gS3h4uWLGJ2yhaPxOtwlzrFKvvrbWZ-","ns":""},"count":1},"inc_marker":{"position":"00000000002.6.10"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:47:58.097 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-12 2021-03-11T00:47:58.098 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-12', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:58.099 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-12', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:58.099 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-12 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:58.148 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:58.148 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:58.344 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":"00000000011.27.5"},{"key":2,"val":""},{"key":3,"val":"00000000003.15.5"},{"key":4,"val":""},{"key":5,"val":"00000000002.6.10"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:47:58.345 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-12 2021-03-11T00:47:58.345 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-12', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:58.346 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-12', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:58.346 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-12 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:58.394 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:58.395 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:58.651 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000011.27.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000003.15.5"}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.6.10"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:47:58.651 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-12 2021-03-11T00:47:58.652 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-12', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:58.652 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-12', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:58.653 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-12 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:58.701 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:58.701 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:58.882 INFO:teuthology.orchestra.run.smithi124.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":"00000000003.12.5"},{"key":2,"val":""},{"key":3,"val":"00000000011.41.5"},{"key":4,"val":""},{"key":5,"val":"00000000003.11.9"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:47:58.883 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-12 2021-03-11T00:47:58.884 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-12', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:58.884 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-12', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:47:58.885 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-12 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:47:58.927 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:47:58.928 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:47:59.134 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj-test-zone1","instance":"95K-tIYWIOr8Q8PhyztL3fZpDaVRQ3N","ns":""},"count":1},"inc_marker":{"position":"00000000003.12.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj-test-zone2","instance":"IoGVAznPywpWoDPeunnFo6cMKKGv4-A","ns":""},"count":1},"inc_marker":{"position":"00000000011.41.5"}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"gS3h4uWLGJ2yhaPxOtwlzrFKvvrbWZ-","ns":""},"count":1},"inc_marker":{"position":"00000000003.11.9"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:47:59.135 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-12 2021-03-11T00:47:59.135 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-12 zones={test-zone2, test-zone1} 2021-03-11T00:47:59.145 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:47:59.195 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2021-03-11T00:47:59.196 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2021-03-11T00:47:59.196 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2021-03-11T00:47:59.197 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:47:59.203 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2021-03-11T00:47:59.203 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2021-03-11T00:47:59.204 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2021-03-11T00:47:59.261 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2021-03-11T00:47:59.272 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2021-03-11T00:47:59.344 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2021-03-11T00:47:59.404 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2021-03-11T00:47:59.463 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2021-03-11T00:47:59.586 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2021-03-11T00:47:59.692 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2021-03-11T00:47:59.747 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2021-03-11T00:47:59.862 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2021-03-11T00:47:59.968 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-12 zones={test-zone2, test-zone1} 2021-03-11T00:47:59.969 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_versioned_object_incremental_sync ... ok 2021-03-11T00:48:00.008 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:00.009 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:00.010 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:00.063 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:00.063 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:00.217 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=470410ef-8c66-4339-a004-064d3e2c6c91 2021-03-11T00:48:00.245 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615423680.010411_137.1","last_update":"2021-03-11T00:48:00.010411Z"},{"marker":"1_1615423619.033309_101.1","last_update":"2021-03-11T00:46:59.033309Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T00:48:00.246 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2021-03-11T00:48:00.246 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:00.247 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:00.248 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:00.288 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:00.288 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:00.455 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423680.010411_137.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:48:00.010411Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:48:00.456 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2021-03-11T00:48:00.456 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T00:48:00.457 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:00.457 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:00.458 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:00.538 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:00.539 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:00.798 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423680.010411_137.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:48:00.010411Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:48:00.799 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T00:48:00.851 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:00.851 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:00.852 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-13 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:00.893 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:00.893 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:01.087 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":"00000000002.10.10"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:48:01.088 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-13 2021-03-11T00:48:01.088 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-13', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:01.089 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-13', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:01.089 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-13 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:01.134 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:01.135 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:01.318 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:48:01.319 WARNING:rgw_multi.tests:bucket nolznk-13 zone test-zone2 behind zone test-zone1: shard=5 master=00000000002.10.10 target= 2021-03-11T00:48:06.321 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-13', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:06.321 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-13', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:06.322 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-13 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:06.368 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:06.368 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:06.541 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"-ussSJ0sJpeECNsVzUBMrvPkZZILMpv","ns":""},"count":1},"inc_marker":{"position":"00000000003.12.9"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:48:06.542 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-13 2021-03-11T00:48:06.542 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:06.543 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:06.543 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-13 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:06.585 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:06.585 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:06.778 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":"00000000003.12.9"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:48:06.779 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-13 2021-03-11T00:48:06.779 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-13', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:06.780 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-13', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:06.781 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-13 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:06.825 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:06.825 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:07.077 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000003.12.9"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:48:07.078 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-13 2021-03-11T00:48:07.078 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-13', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:07.079 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-13', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:07.079 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-13 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:07.157 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:07.157 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:07.337 INFO:teuthology.orchestra.run.smithi124.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":"00000000002.8.10"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:48:07.337 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-13 2021-03-11T00:48:07.338 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:07.339 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:07.339 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-13 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:07.382 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:07.383 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:07.590 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"-ussSJ0sJpeECNsVzUBMrvPkZZILMpv","ns":""},"count":1},"inc_marker":{"position":"00000000002.8.10"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:48:07.591 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-13 2021-03-11T00:48:07.591 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-13 zones={test-zone2, test-zone1} 2021-03-11T00:48:07.601 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:48:07.648 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2021-03-11T00:48:07.648 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:48:07.654 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2021-03-11T00:48:07.708 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2021-03-11T00:48:07.719 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2021-03-11T00:48:07.783 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2021-03-11T00:48:07.844 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-13 zones={test-zone2, test-zone1} 2021-03-11T00:48:08.827 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:08.901 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:08.902 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-13 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:08.961 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:08.961 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:09.144 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":"00000000003.12.9"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000020.42.11"},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:48:09.145 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-13 2021-03-11T00:48:09.145 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-13', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:09.146 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-13', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:09.147 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-13 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:09.191 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:09.192 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:09.362 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"-ussSJ0sJpeECNsVzUBMrvPkZZILMpv","ns":""},"count":1},"inc_marker":{"position":"00000000003.12.9"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"xPdV0p2-AVsNFaI5NEiffEdlySjRhGq","ns":""},"count":4},"inc_marker":{"position":"00000000020.42.11"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:48:09.362 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-13 2021-03-11T00:48:09.363 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:09.363 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:09.364 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-13 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:09.406 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:09.406 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:09.586 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":"00000000003.12.9"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000032.86.5"},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:48:09.586 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-13 2021-03-11T00:48:09.587 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-13', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:09.588 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-13', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:09.588 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-13 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:09.634 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:09.634 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:09.892 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000003.12.9"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000020.42.11"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:48:09.892 WARNING:rgw_multi.tests:bucket nolznk-13 zone test-zone3 behind zone test-zone1: shard=8 master=00000000032.86.5 target=00000000020.42.11 2021-03-11T00:48:14.894 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-13', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:14.895 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-13', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:14.895 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-13 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:14.940 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:14.941 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:15.193 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000003.12.9"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000032.86.5"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:48:15.194 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-13 2021-03-11T00:48:15.195 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-13', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:15.195 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-13', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:15.196 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-13 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:15.279 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:15.279 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:15.457 INFO:teuthology.orchestra.run.smithi124.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":"00000000002.8.10"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000031.88.5"},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:48:15.457 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-13 2021-03-11T00:48:15.458 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:15.459 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:15.459 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-13 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:15.503 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:15.503 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:15.693 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"-ussSJ0sJpeECNsVzUBMrvPkZZILMpv","ns":""},"count":1},"inc_marker":{"position":"00000000002.8.10"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"xPdV0p2-AVsNFaI5NEiffEdlySjRhGq","ns":""},"count":8},"inc_marker":{"position":"00000000031.88.5"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:48:15.693 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-13 2021-03-11T00:48:15.694 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-13 zones={test-zone2, test-zone1} 2021-03-11T00:48:15.703 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:48:15.711 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2021-03-11T00:48:15.712 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.712 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.713 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.713 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.714 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.714 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.715 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.715 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.715 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.716 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.717 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.717 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.717 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.718 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.718 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.719 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.719 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.720 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.720 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.720 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.721 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:48:15.730 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2021-03-11T00:48:15.730 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.731 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.731 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.732 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.732 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.733 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.733 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.734 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.734 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.735 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.735 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.735 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.736 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.736 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.737 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.737 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.738 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.738 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.739 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.739 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:15.756 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2021-03-11T00:48:15.769 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2021-03-11T00:48:15.837 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2021-03-11T00:48:15.900 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:15.955 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:16.079 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:16.188 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:16.243 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:16.365 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:16.473 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:16.527 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:16.639 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:16.783 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:16.840 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:16.957 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:17.070 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:17.124 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:17.242 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:17.346 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:17.405 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:17.519 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:17.625 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:17.679 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:17.801 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:17.905 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:17.959 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:18.073 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:18.180 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:18.235 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:18.355 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:18.460 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:18.515 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:18.631 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:18.740 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:18.795 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:18.912 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:19.016 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:19.075 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:19.189 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:19.293 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:19.348 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:19.473 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:19.582 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:19.639 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:19.753 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:19.856 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:19.912 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:20.025 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:20.129 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:20.197 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:20.316 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:20.424 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:20.479 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:20.601 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:20.708 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:20.764 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:20.886 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:20.992 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:21.047 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:21.180 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:21.285 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:21.342 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:21.459 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:21.565 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-13 zones={test-zone2, test-zone1} 2021-03-11T00:48:21.566 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:21.567 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:21.567 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:21.604 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:21.605 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:21.804 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615423685.919484_54.3","last_update":"2021-03-11T00:48:05.919484Z"},{"marker":"1_1615423611.007858_87.1","last_update":"2021-03-11T00:46:51.007858Z"},{"marker":"1_1615423685.919484_83.2","last_update":"2021-03-11T00:48:05.919484Z"},{"marker":"1_1615423687.865298_89.1","last_update":"2021-03-11T00:48:07.865298Z"}] 2021-03-11T00:48:21.805 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2021-03-11T00:48:21.806 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:21.806 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:21.807 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:21.854 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:21.854 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:22.029 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17085014794446772242},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_54.3","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423611.007858_87.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:51.007858Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_83.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423687.865298_89.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:07.865298Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:48:22.030 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2021-03-11T00:48:22.031 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:22.032 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:22.032 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:22.073 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:22.074 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:22.272 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615423685.919484_54.3","last_update":"2021-03-11T00:48:05.919484Z"},{"marker":"1_1615423611.007858_87.1","last_update":"2021-03-11T00:46:51.007858Z"},{"marker":"1_1615423685.919484_83.2","last_update":"2021-03-11T00:48:05.919484Z"},{"marker":"1_1615423687.865298_89.1","last_update":"2021-03-11T00:48:07.865298Z"}] 2021-03-11T00:48:22.273 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone3 source_zone=test-zone1 2021-03-11T00:48:22.274 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:22.275 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:22.276 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:22.319 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:22.319 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:22.579 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":6834771546787753244},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_54.3","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423611.007858_87.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:51.007858Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_83.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423687.865298_89.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:07.865298Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:48:22.580 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone3 source_zone=test-zone1 2021-03-11T00:48:22.580 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:22.581 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:22.582 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:22.661 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:22.662 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:22.833 INFO:teuthology.orchestra.run.smithi124.stdout:[{"marker":"1_1615423686.841255_28.3","last_update":"2021-03-11T00:48:06.841255Z"},{"marker":"1_1615423609.883637_86.1","last_update":"2021-03-11T00:46:49.883637Z"},{"marker":"1_1615423686.841255_82.2","last_update":"2021-03-11T00:48:06.841255Z"},{"marker":"1_1615423687.912680_80.1","last_update":"2021-03-11T00:48:07.912680Z"}] 2021-03-11T00:48:22.834 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2021-03-11T00:48:22.835 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:22.836 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:22.837 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:22.881 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:22.882 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:23.071 INFO:teuthology.orchestra.run.smithi119.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":3384081757615347012},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615423686.841255_28.3","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:06.841255Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423609.883637_86.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:49.883637Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423686.841255_82.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:06.841255Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423687.912680_80.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:07.912680Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:48:23.073 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2021-03-11T00:48:23.074 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_concurrent_versioned_object_incremental_sync ... ok 2021-03-11T00:48:23.103 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:23.103 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:23.104 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:23.150 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:23.151 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:23.310 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=470410ef-8c66-4339-a004-064d3e2c6c91 2021-03-11T00:48:23.339 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615423703.106524_151.1","last_update":"2021-03-11T00:48:23.106524Z"},{"marker":"1_1615423619.033309_101.1","last_update":"2021-03-11T00:46:59.033309Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T00:48:23.339 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2021-03-11T00:48:23.340 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:23.340 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:23.341 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:23.385 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:23.385 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:23.548 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423680.010411_137.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:48:00.010411Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:48:23.549 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423703.106524_151.1 target=1_1615423680.010411_137.1 2021-03-11T00:48:28.551 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:28.562 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:28.565 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:28.633 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:28.634 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:28.813 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423703.106524_151.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:48:23.106524Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:48:28.814 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2021-03-11T00:48:28.815 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T00:48:28.815 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:28.816 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:28.816 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:28.923 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:28.924 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:29.180 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423703.106524_151.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:48:23.106524Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:48:29.181 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T00:48:29.241 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-14', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:29.242 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-14', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:29.243 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-14 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:29.299 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:29.300 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:29.485 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000001.7.5"},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:48:29.486 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-14 2021-03-11T00:48:29.487 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:29.488 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:29.489 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:29.548 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:29.548 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:29.736 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:48:29.737 WARNING:rgw_multi.tests:bucket nolznk-14 zone test-zone2 behind zone test-zone1: shard=8 master=00000000001.7.5 target= 2021-03-11T00:48:34.738 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:34.739 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:34.740 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:34.783 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:34.783 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:34.963 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.7.5"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:48:34.964 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-14 2021-03-11T00:48:34.964 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-14', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:34.965 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-14', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:34.965 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-14 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:35.008 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:35.009 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:35.186 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000001.7.5"},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:48:35.186 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-14 2021-03-11T00:48:35.187 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:35.188 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:35.188 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:35.234 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:35.234 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:35.492 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.7.5"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:48:35.493 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-14 2021-03-11T00:48:35.494 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:35.495 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:35.496 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:35.572 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:35.573 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:35.745 INFO:teuthology.orchestra.run.smithi124.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000001.9.5"},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:48:35.746 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-14 2021-03-11T00:48:35.747 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-14', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:35.747 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-14', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:35.748 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-14 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:35.793 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:35.793 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:35.979 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.9.5"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:48:35.981 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-14 2021-03-11T00:48:35.982 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-14 zones={test-zone2, test-zone1} 2021-03-11T00:48:36.001 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:48:36.051 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:36.053 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:48:36.059 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:36.117 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:36.129 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:36.195 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:36.252 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-14 zones={test-zone2, test-zone1} 2021-03-11T00:48:36.265 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:36.265 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:36.266 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:36.304 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:36.304 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:36.453 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=470410ef-8c66-4339-a004-064d3e2c6c91 2021-03-11T00:48:36.483 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615423716.269301_157.1","last_update":"2021-03-11T00:48:36.269301Z"},{"marker":"1_1615423619.033309_101.1","last_update":"2021-03-11T00:46:59.033309Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T00:48:36.483 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2021-03-11T00:48:36.484 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:36.484 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:36.485 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:36.523 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:36.524 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:36.698 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423703.106524_151.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:48:23.106524Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:48:36.699 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423716.269301_157.1 target=1_1615423703.106524_151.1 2021-03-11T00:48:41.700 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:41.701 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:41.702 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:41.742 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:41.743 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:41.914 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423716.269301_157.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:48:36.269301Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:48:41.915 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2021-03-11T00:48:41.915 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T00:48:41.916 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:41.916 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:41.918 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:41.999 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:41.999 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:42.356 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423716.269301_157.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:48:36.269301Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:48:42.357 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T00:48:42.407 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-14', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:42.408 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-14', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:42.409 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-14 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:42.449 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:42.450 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:42.647 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000003.10.12"},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:48:42.648 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-14 2021-03-11T00:48:42.648 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:42.649 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:42.650 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:42.698 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:42.698 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:42.879 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.7.5"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:48:42.880 WARNING:rgw_multi.tests:bucket nolznk-14 zone test-zone2 behind zone test-zone1: shard=8 master=00000000003.10.12 target=00000000001.7.5 2021-03-11T00:48:47.881 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:47.881 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:47.882 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:47.928 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:47.928 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:48.123 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000003.10.12"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:48:48.124 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-14 2021-03-11T00:48:48.125 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-14', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:48.125 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-14', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:48.126 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-14 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:48.168 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:48.169 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:48.365 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000003.10.12"},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:48:48.365 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-14 2021-03-11T00:48:48.366 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:48.367 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:48.367 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:48.412 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:48.413 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:48.666 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000003.10.12"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:48:48.666 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-14 2021-03-11T00:48:48.667 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:48.668 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:48.668 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:48.751 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:48.751 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:48.938 INFO:teuthology.orchestra.run.smithi124.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000003.14.12"},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:48:48.939 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-14 2021-03-11T00:48:48.939 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-14', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:48.940 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-14', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:48.941 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-14 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:48.986 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:48.986 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:49.169 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000003.14.12"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:48:49.169 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-14 2021-03-11T00:48:49.170 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-14 zones={test-zone2, test-zone1} 2021-03-11T00:48:49.179 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:48:49.227 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:49.228 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:49.229 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:48:49.239 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:49.240 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:48:49.292 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:49.303 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:49.378 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:49.449 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:49.464 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:49.537 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:48:49.598 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-14 zones={test-zone2, test-zone1} 2021-03-11T00:48:49.611 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:49.611 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:49.612 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:49.651 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:49.652 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:49.793 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=470410ef-8c66-4339-a004-064d3e2c6c91 2021-03-11T00:48:49.822 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615423729.613265_163.1","last_update":"2021-03-11T00:48:49.613265Z"},{"marker":"1_1615423619.033309_101.1","last_update":"2021-03-11T00:46:59.033309Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T00:48:49.822 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2021-03-11T00:48:49.823 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:49.823 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:49.824 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:49.863 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:49.864 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:50.045 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423716.269301_157.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:48:36.269301Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:48:50.046 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423729.613265_163.1 target=1_1615423716.269301_157.1 2021-03-11T00:48:55.048 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:55.067 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:55.068 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:55.108 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:55.108 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:55.285 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423729.613265_163.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:48:49.613265Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:48:55.286 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2021-03-11T00:48:55.286 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T00:48:55.287 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:55.287 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:55.288 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:55.370 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:55.370 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:55.629 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423729.613265_163.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:48:49.613265Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:48:55.630 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T00:48:55.679 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-14', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:55.679 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-14', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:55.680 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-14 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:55.719 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:55.720 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:55.900 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000005.14.17"},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:48:55.900 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-14 2021-03-11T00:48:55.901 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:55.902 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:48:55.902 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:48:55.948 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:48:55.949 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:48:56.167 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000003.10.12"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:48:56.167 WARNING:rgw_multi.tests:bucket nolznk-14 zone test-zone2 behind zone test-zone1: shard=8 master=00000000005.14.17 target=00000000003.10.12 2021-03-11T00:49:01.169 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:01.178 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:01.178 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:01.218 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:01.218 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:01.427 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000005.14.17"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:49:01.428 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-14 2021-03-11T00:49:01.428 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-14', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:01.429 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-14', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:01.429 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-14 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:01.471 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:01.471 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:01.648 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000005.14.17"},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:49:01.649 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-14 2021-03-11T00:49:01.649 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:01.650 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:01.651 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:01.695 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:01.696 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:01.949 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000005.14.17"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:49:01.950 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-14 2021-03-11T00:49:01.951 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:01.951 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:01.952 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:02.032 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:02.032 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:02.228 INFO:teuthology.orchestra.run.smithi124.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000005.20.17"},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:49:02.229 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-14 2021-03-11T00:49:02.229 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-14', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:02.230 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-14', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:02.231 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-14 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:02.280 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:02.280 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:02.472 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000005.20.17"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:49:02.473 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-14 2021-03-11T00:49:02.473 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-14 zones={test-zone2, test-zone1} 2021-03-11T00:49:02.518 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:49:02.598 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:49:02.598 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:49:02.599 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:49:02.615 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:49:02.616 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:49:02.695 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:49:02.736 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:49:02.860 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:49:02.969 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:49:03.033 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:49:03.155 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:49:03.247 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-14 zones={test-zone2, test-zone1} 2021-03-11T00:49:03.248 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_version_suspended_incremental_sync ... ok 2021-03-11T00:49:03.249 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=nolznk-15 2021-03-11T00:49:03.276 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=nolznk-16 2021-03-11T00:49:03.361 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:03.362 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:03.363 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:03.406 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:03.407 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:03.561 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=470410ef-8c66-4339-a004-064d3e2c6c91 2021-03-11T00:49:03.605 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615423743.357959_179.1","last_update":"2021-03-11T00:49:03.357959Z"},{"marker":"1_1615423619.033309_101.1","last_update":"2021-03-11T00:46:59.033309Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T00:49:03.608 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2021-03-11T00:49:03.609 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:03.612 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:03.614 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:03.663 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:03.663 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:03.849 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423729.613265_163.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:48:49.613265Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:49:03.850 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423743.357959_179.1 target=1_1615423729.613265_163.1 2021-03-11T00:49:08.851 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:08.860 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:08.861 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:08.901 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:08.901 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:09.101 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423743.357959_179.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:49:03.357959Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:49:09.102 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2021-03-11T00:49:09.103 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T00:49:09.104 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:09.105 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:09.105 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:09.184 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:09.185 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:09.447 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423743.357959_179.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:49:03.357959Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:49:09.448 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T00:49:09.577 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-15', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:09.578 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-15', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:09.578 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-15 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:09.621 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:09.621 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:09.793 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000003.14.11"},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:49:09.794 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-15 2021-03-11T00:49:09.794 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-15', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:09.795 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-15', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:09.795 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-15 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:09.838 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:09.839 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:10.048 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:49:10.049 WARNING:rgw_multi.tests:bucket nolznk-15 zone test-zone2 behind zone test-zone1: shard=8 master=00000000003.14.11 target= 2021-03-11T00:49:15.051 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-15', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:15.052 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-15', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:15.052 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-15 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:15.097 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:15.097 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:15.297 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"H9GfRp1WTaNE2QEele15bsF.tv85u3d","ns":""},"count":2},"inc_marker":{"position":"00000000003.14.11"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:49:15.298 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-15 2021-03-11T00:49:15.298 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-15', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:15.299 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-15', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:15.300 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-15 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:15.342 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:15.343 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:15.520 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000003.14.11"},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:49:15.521 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-15 2021-03-11T00:49:15.521 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-15', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:15.522 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-15', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:15.522 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-15 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:15.564 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:15.564 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:15.819 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000003.14.11"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:49:15.819 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-15 2021-03-11T00:49:15.820 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-15', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:15.821 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-15', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:15.821 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-15 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:15.902 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:15.902 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:16.097 INFO:teuthology.orchestra.run.smithi124.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000002.20.5"},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:49:16.097 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-15 2021-03-11T00:49:16.098 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-15', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:16.098 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-15', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:16.099 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-15 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:16.143 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:16.144 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:16.335 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"30WBJ8Y04ULxjRND3sm2ESxOji6QzIx","ns":""},"count":1},"inc_marker":{"position":"00000000002.20.5"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:49:16.336 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-15 2021-03-11T00:49:16.337 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-15 zones={test-zone2, test-zone1} 2021-03-11T00:49:16.357 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:49:16.407 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:49:16.407 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:49:16.408 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:49:16.414 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:49:16.415 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:49:16.468 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:49:16.476 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:49:16.486 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:49:16.550 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-15 zones={test-zone2, test-zone1} 2021-03-11T00:49:16.551 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-16', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:16.552 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-16', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:16.552 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-16 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:16.592 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:16.592 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:16.754 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000002.12.5"},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:49:16.755 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-16 2021-03-11T00:49:16.756 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-16', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:16.756 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-16', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:16.757 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-16 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:16.800 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:16.801 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:16.999 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"ys93NiI4QHvaHFufk-SRxoWPUGDSYOe","ns":""},"count":2},"inc_marker":{"position":"00000000002.12.5"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:49:17.000 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-16 2021-03-11T00:49:17.001 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-16', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:17.001 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-16', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:17.002 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-16 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:17.043 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:17.043 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:17.220 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000002.12.5"},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:49:17.221 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-16 2021-03-11T00:49:17.221 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-16', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:17.222 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-16', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:17.223 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-16 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:17.270 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:17.271 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:17.525 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.12.5"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:49:17.526 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-16 2021-03-11T00:49:17.527 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-16', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:17.528 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-16', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:17.529 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-16 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:17.570 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:17.570 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:17.749 INFO:teuthology.orchestra.run.smithi124.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000003.22.11"},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:49:17.750 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-16 2021-03-11T00:49:17.751 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-16', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:17.751 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-16', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:17.752 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-16 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:17.797 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:17.797 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:17.975 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"ys93NiI4QHvaHFufk-SRxoWPUGDSYOe","ns":""},"count":2},"inc_marker":{"position":"00000000003.22.11"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:49:17.976 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-16 2021-03-11T00:49:17.977 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-16 zones={test-zone2, test-zone1} 2021-03-11T00:49:17.988 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:49:18.041 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:49:18.042 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:49:18.043 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:49:18.053 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:49:18.054 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:49:18.136 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:49:18.147 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:49:18.190 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:49:18.326 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-16 zones={test-zone2, test-zone1} 2021-03-11T00:49:18.327 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_delete_marker_full_sync ... ok 2021-03-11T00:49:18.331 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=nolznk-17 2021-03-11T00:49:18.368 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=nolznk-18 2021-03-11T00:49:18.498 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:18.499 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:18.500 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:18.546 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:18.546 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:18.694 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=470410ef-8c66-4339-a004-064d3e2c6c91 2021-03-11T00:49:18.722 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615423758.490731_202.1","last_update":"2021-03-11T00:49:18.490731Z"},{"marker":"1_1615423619.033309_101.1","last_update":"2021-03-11T00:46:59.033309Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T00:49:18.723 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2021-03-11T00:49:18.723 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:18.724 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:18.724 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:18.778 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:18.778 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:18.960 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423743.357959_179.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:49:03.357959Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:49:18.960 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423758.490731_202.1 target=1_1615423743.357959_179.1 2021-03-11T00:49:23.965 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:23.974 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:23.975 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:24.014 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:24.015 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:24.194 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423758.490731_202.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:49:18.490731Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:49:24.195 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2021-03-11T00:49:24.196 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T00:49:24.196 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:24.197 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:24.197 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:24.275 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:24.275 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:24.528 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423758.490731_202.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:49:18.490731Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:49:24.529 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T00:49:24.675 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-17', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:24.675 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-17', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:24.676 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-17 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:24.717 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:24.718 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:24.901 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000003.10.6"},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:49:24.901 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-17 2021-03-11T00:49:24.902 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:24.903 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:24.903 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:24.947 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:24.947 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:25.138 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:49:25.139 WARNING:rgw_multi.tests:bucket nolznk-17 zone test-zone2 behind zone test-zone1: shard=8 master=00000000003.10.6 target= 2021-03-11T00:49:30.140 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:30.141 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:30.141 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:30.188 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:30.188 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:30.379 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"null","ns":""},"count":1},"inc_marker":{"position":"00000000003.10.6"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:49:30.379 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-17 2021-03-11T00:49:30.380 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-17', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:30.380 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-17', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:30.381 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-17 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:30.422 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:30.423 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:30.608 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000003.10.6"},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:49:30.608 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-17 2021-03-11T00:49:30.609 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:30.610 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:30.610 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:30.659 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:30.660 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:30.919 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000003.10.6"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:49:30.920 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-17 2021-03-11T00:49:30.920 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:30.921 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:30.921 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:31.000 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:31.001 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:31.182 INFO:teuthology.orchestra.run.smithi124.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000001.18.12"},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:49:31.182 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-17 2021-03-11T00:49:31.183 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-17', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:31.183 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-17', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:31.184 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-17 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:31.228 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:31.228 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:31.416 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"null","ns":""},"count":1},"inc_marker":{"position":"00000000001.18.12"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:49:31.417 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-17 2021-03-11T00:49:31.417 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-17 zones={test-zone2, test-zone1} 2021-03-11T00:49:31.436 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:49:31.483 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:49:31.483 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:49:31.489 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:49:31.544 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:49:31.551 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-17 zones={test-zone2, test-zone1} 2021-03-11T00:49:31.552 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-18', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:31.552 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-18', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:31.553 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-18 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:31.592 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:31.592 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:31.777 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000001.8.12"},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:49:31.777 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-18 2021-03-11T00:49:31.778 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-18', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:31.778 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-18', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:31.779 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-18 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:31.822 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:31.823 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:32.009 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"null","ns":""},"count":1},"inc_marker":{"position":"00000000001.8.12"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:49:32.010 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-18 2021-03-11T00:49:32.011 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-18', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:32.011 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-18', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:32.012 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-18 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:32.055 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:32.055 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:32.236 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000001.8.12"},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:49:32.236 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-18 2021-03-11T00:49:32.237 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-18', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:32.237 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-18', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:32.238 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-18 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:32.284 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:32.285 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:32.538 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.8.12"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:49:32.539 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-18 2021-03-11T00:49:32.540 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-18', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:32.541 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-18', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:32.542 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-18 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:32.628 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:32.628 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:32.811 INFO:teuthology.orchestra.run.smithi124.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000003.18.6"},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:49:32.811 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-18 2021-03-11T00:49:32.812 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-18', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:32.812 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-18', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:32.813 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-18 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:32.857 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:32.858 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:33.045 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"null","ns":""},"count":1},"inc_marker":{"position":"00000000003.18.6"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:49:33.045 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-18 2021-03-11T00:49:33.046 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-18 zones={test-zone2, test-zone1} 2021-03-11T00:49:33.055 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:49:33.103 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:49:33.103 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:49:33.111 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2021-03-11T00:49:33.164 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2021-03-11T00:49:33.170 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-18 zones={test-zone2, test-zone1} 2021-03-11T00:49:33.170 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_suspended_delete_marker_full_sync ... ok 2021-03-11T00:49:33.174 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=nolznk-19 2021-03-11T00:49:33.204 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=nolznk-20 2021-03-11T00:49:33.374 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_versioning ... ok 2021-03-11T00:49:33.375 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=nolznk-21 2021-03-11T00:49:33.401 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=nolznk-22 2021-03-11T00:49:33.577 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_acl ... ok 2021-03-11T00:49:33.578 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=nolznk-23 2021-03-11T00:49:33.608 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=nolznk-24 2021-03-11T00:49:34.796 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_cors ... ok 2021-03-11T00:49:34.797 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=nolznk-25 2021-03-11T00:49:34.823 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=nolznk-26 2021-03-11T00:49:34.871 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:34.872 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:34.872 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:34.921 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:34.922 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:35.096 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=470410ef-8c66-4339-a004-064d3e2c6c91 2021-03-11T00:49:35.118 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615423774.849040_253.1","last_update":"2021-03-11T00:49:34.849040Z"},{"marker":"1_1615423619.033309_101.1","last_update":"2021-03-11T00:46:59.033309Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T00:49:35.118 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2021-03-11T00:49:35.119 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:35.120 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:35.120 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:35.163 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:35.164 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:35.341 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423774.849040_253.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:49:34.849040Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:49:35.342 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2021-03-11T00:49:35.343 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T00:49:35.343 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:35.344 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:35.344 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:35.430 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:35.430 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:35.685 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423774.849040_253.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:49:34.849040Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:49:35.686 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T00:49:35.949 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_delete_notempty ... ok 2021-03-11T00:49:35.952 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=nolznk-27 2021-03-11T00:49:35.979 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=nolznk-28 2021-03-11T00:49:36.027 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:36.028 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:36.029 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:36.074 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:36.075 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:36.223 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=470410ef-8c66-4339-a004-064d3e2c6c91 2021-03-11T00:49:36.252 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615423776.006942_263.1","last_update":"2021-03-11T00:49:36.006942Z"},{"marker":"1_1615423619.033309_101.1","last_update":"2021-03-11T00:46:59.033309Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T00:49:36.253 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2021-03-11T00:49:36.253 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:36.254 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:36.254 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:36.296 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:36.297 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:36.464 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423776.006942_263.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:49:36.006942Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:49:36.465 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2021-03-11T00:49:36.466 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T00:49:36.466 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:36.467 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:49:36.467 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:49:36.544 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:36.545 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:36.799 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"470410ef-8c66-4339-a004-064d3e2c6c91","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423776.006942_263.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:49:36.006942Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:49:36.799 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T00:49:36.800 DEBUG:tasks.rgw.c2.client.1:waiting for process to exit 2021-03-11T00:49:36.801 INFO:teuthology.orchestra.run:waiting for 300 2021-03-11T00:49:36.844 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T00:49:36.840+0000 7fc66d7c0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term radosgw --rgw-frontends beast port=8001 -n client.1 --cluster c2 -k /etc/ceph/c2.client.1.keyring --log-file /var/log/ceph/rgw.c2.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.1.sock --foreground --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm (PID: 18133) UID: 0 2021-03-11T00:49:36.845 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T00:49:36.840+0000 7fc684e830c0 -1 shutting down 2021-03-11T00:49:39.298 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.1 is failed for ~0s 2021-03-11T00:49:42.904 INFO:tasks.rgw.c2.client.1:Stopped 2021-03-11T00:49:42.906 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--master', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:49:42.907 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--master', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:49:42.908 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone modify --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --master --cluster c2 --debug-rgw 0 --debug-ms 0 2021-03-11T00:49:42.948 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:42.949 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:42.976 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:49:42.972+0000 7f1a97455080 0 NOTICE: overriding master zone: 4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0 2021-03-11T00:49:42.977 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:49:42.972+0000 7f1a97455080 0 failed reading obj info from .rgw.root:zone_info.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0: (2) No such file or directory 2021-03-11T00:49:42.977 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:49:42.972+0000 7f1a97455080 0 WARNING: could not read zone params for zone id=4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0 name=test-zone1 2021-03-11T00:49:42.990 INFO:teuthology.orchestra.run.smithi124.stdout:{"id":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","name":"test-zone2","domain_root":"test-zone2.rgw.meta:root","control_pool":"test-zone2.rgw.control","gc_pool":"test-zone2.rgw.log:gc","lc_pool":"test-zone2.rgw.log:lc","log_pool":"test-zone2.rgw.log","intent_log_pool":"test-zone2.rgw.log:intent","usage_log_pool":"test-zone2.rgw.log:usage","roles_pool":"test-zone2.rgw.meta:roles","reshard_pool":"test-zone2.rgw.log:reshard","user_keys_pool":"test-zone2.rgw.meta:users.keys","user_email_pool":"test-zone2.rgw.meta:users.email","user_swift_pool":"test-zone2.rgw.meta:users.swift","user_uid_pool":"test-zone2.rgw.meta:users.uid","otp_pool":"test-zone2.rgw.otp","system_key":{"access_key":"8WRXV4QK6AZ20SFM","secret_key":"q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone2.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone2.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone2.rgw.buckets.non-ec","index_type":0}}],"realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5"} 2021-03-11T00:49:42.991 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:49:42.991 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:49:42.992 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2021-03-11T00:49:43.071 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:49:43.071 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:49:43.305 INFO:teuthology.orchestra.run.smithi124.stdout:{"id":"60afeec2-94d4-4f5a-a646-d47261804cd8","epoch":1,"predecessor_uuid":"470410ef-8c66-4339-a004-064d3e2c6c91","sync_status":["1_1615423776.006942_263.1","1_1615423619.033309_101.1","",""],"period_map":{"id":"60afeec2-94d4-4f5a-a646-d47261804cd8","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","zones":[{"id":"2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05","name":"test-zone3","endpoints":["http://smithi124.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","name":"test-zone1","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","name":"test-zone2","endpoints":["http://smithi124.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","sync_policy":{"groups":[]}}],"short_zone_ids":[{"key":"2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05","val":906122584},{"key":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","val":3593284871},{"key":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","val":1094358664}]},"master_zonegroup":"test-zonegroup","master_zone":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","realm_name":"test-realm","realm_epoch":3} 2021-03-11T00:49:43.306 INFO:rgw_multi.tests:Set master zone=test-zone2, waiting 60s for reconfiguration.. 2021-03-11T00:49:44.905 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.1 has been restored 2021-03-11T00:50:43.308 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=nolznk-29 2021-03-11T00:50:43.368 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=nolznk-30 2021-03-11T00:50:43.400 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:50:43.400 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:50:43.401 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:50:43.440 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:50:43.441 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:50:43.586 INFO:teuthology.orchestra.run.smithi124.stderr:No --period given, using current period=60afeec2-94d4-4f5a-a646-d47261804cd8 2021-03-11T00:50:43.617 INFO:teuthology.orchestra.run.smithi124.stdout:[{"marker":"1_1615423843.392949_260.1","last_update":"2021-03-11T00:50:43.392949Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T00:50:43.617 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone1 2021-03-11T00:50:43.618 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:50:43.618 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:50:43.619 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:50:43.659 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:50:43.659 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:50:43.841 INFO:teuthology.orchestra.run.smithi119.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"60afeec2-94d4-4f5a-a646-d47261804cd8","realm_epoch":3},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":55,"pos":0,"timestamp":"0.000000","realm_epoch":3}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":3}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":3}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":3}}]},"full_sync":{"total":55,"complete":55}} 2021-03-11T00:50:43.842 WARNING:rgw_multi.tests:zone test-zone1 behind master: shard=0 master=1_1615423843.392949_260.1 target= 2021-03-11T00:50:48.843 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:50:48.849 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:50:48.850 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:50:48.890 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:50:48.891 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:50:49.093 INFO:teuthology.orchestra.run.smithi119.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"60afeec2-94d4-4f5a-a646-d47261804cd8","realm_epoch":3},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423843.392949_260.1","next_step_marker":"","total_entries":55,"pos":0,"timestamp":"2021-03-11T00:50:43.392949Z","realm_epoch":3}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":3}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":3}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":3}}]},"full_sync":{"total":55,"complete":55}} 2021-03-11T00:50:49.094 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone1 2021-03-11T00:50:49.094 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'modify', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--master', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:50:49.103 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'modify', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--master', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:50:49.104 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone modify --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --master --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T00:50:49.173 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:50:49.173 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:50:49.199 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:50:49.194+0000 7f77cce53080 0 NOTICE: overriding master zone: a2f6d1db-e7f6-4c80-8390-898891b9eb91 2021-03-11T00:50:49.200 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:50:49.194+0000 7f77cce53080 0 failed reading obj info from .rgw.root:zone_info.2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05: (2) No such file or directory 2021-03-11T00:50:49.201 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:50:49.194+0000 7f77cce53080 0 WARNING: could not read zone params for zone id=2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05 name=test-zone3 2021-03-11T00:50:49.201 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:50:49.194+0000 7f77cce53080 0 failed reading obj info from .rgw.root:zone_info.a2f6d1db-e7f6-4c80-8390-898891b9eb91: (2) No such file or directory 2021-03-11T00:50:49.201 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:50:49.194+0000 7f77cce53080 0 WARNING: could not read zone params for zone id=a2f6d1db-e7f6-4c80-8390-898891b9eb91 name=test-zone2 2021-03-11T00:50:49.212 INFO:teuthology.orchestra.run.smithi119.stdout:{"id":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","name":"test-zone1","domain_root":"test-zone1.rgw.meta:root","control_pool":"test-zone1.rgw.control","gc_pool":"test-zone1.rgw.log:gc","lc_pool":"test-zone1.rgw.log:lc","log_pool":"test-zone1.rgw.log","intent_log_pool":"test-zone1.rgw.log:intent","usage_log_pool":"test-zone1.rgw.log:usage","roles_pool":"test-zone1.rgw.meta:roles","reshard_pool":"test-zone1.rgw.log:reshard","user_keys_pool":"test-zone1.rgw.meta:users.keys","user_email_pool":"test-zone1.rgw.meta:users.email","user_swift_pool":"test-zone1.rgw.meta:users.swift","user_uid_pool":"test-zone1.rgw.meta:users.uid","otp_pool":"test-zone1.rgw.otp","system_key":{"access_key":"8WRXV4QK6AZ20SFM","secret_key":"q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone1.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone1.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone1.rgw.buckets.non-ec","index_type":0}}],"realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5"} 2021-03-11T00:50:49.212 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:50:49.213 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:50:49.214 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T00:50:49.293 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:50:49.293 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:50:49.521 INFO:teuthology.orchestra.run.smithi119.stdout:{"id":"76343c7e-8958-4b2a-a007-ca994322980a","epoch":1,"predecessor_uuid":"60afeec2-94d4-4f5a-a646-d47261804cd8","sync_status":["1_1615423843.392949_260.1","","",""],"period_map":{"id":"76343c7e-8958-4b2a-a007-ca994322980a","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","zones":[{"id":"2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05","name":"test-zone3","endpoints":["http://smithi124.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","name":"test-zone1","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","name":"test-zone2","endpoints":["http://smithi124.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","sync_policy":{"groups":[]}}],"short_zone_ids":[{"key":"2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05","val":906122584},{"key":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","val":3593284871},{"key":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","val":1094358664}]},"master_zonegroup":"test-zonegroup","master_zone":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","realm_name":"test-realm","realm_epoch":4} 2021-03-11T00:50:49.522 INFO:rgw_multi.tests:Set master zone=test-zone1, waiting 60s for reconfiguration.. 2021-03-11T00:51:49.523 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=nolznk-31 2021-03-11T00:51:49.556 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=nolznk-32 2021-03-11T00:51:49.606 INFO:tasks.rgw.c2.client.1:Restarting daemon 2021-03-11T00:51:49.637 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8001' -n client.1 --cluster c2 -k /etc/ceph/c2.client.1.keyring --log-file /var/log/ceph/rgw.c2.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.1.sock --foreground --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c2.client.1.stdout 2>&1 2021-03-11T00:51:49.642 INFO:tasks.rgw.c2.client.1:Started 2021-03-11T00:51:49.643 DEBUG:teuthology.orchestra.run.smithi124:> curl http://smithi124.front.sepia.ceph.com:8001 2021-03-11T00:51:49.662 INFO:teuthology.orchestra.run.smithi124.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2021-03-11T00:51:49.663 INFO:teuthology.orchestra.run.smithi124.stderr: Dload Upload Total Spent Left Speed 2021-03-11T00:51:49.665 INFO:teuthology.orchestra.run.smithi124.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi124.front.sepia.ceph.com port 8001: Connection refused 2021-03-11T00:51:49.666 DEBUG:teuthology.orchestra.run:got remote process result: 7 2021-03-11T00:51:50.668 DEBUG:teuthology.orchestra.run.smithi124:> curl http://smithi124.front.sepia.ceph.com:8001 2021-03-11T00:51:50.682 INFO:teuthology.orchestra.run.smithi124.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2021-03-11T00:51:50.682 INFO:teuthology.orchestra.run.smithi124.stderr: Dload Upload Total Spent Left Speed 2021-03-11T00:51:50.686 INFO:teuthology.orchestra.run.smithi124.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 134 100 134 0 0 26800 0 --:--:-- --:--:-- --:--:-- 26800 2021-03-11T00:51:50.687 INFO:teuthology.orchestra.run.smithi124.stdout:{"Code":"MethodNotAllowed","RequestId":"tx000000000000000000001-00604969a6-1437-test-zone3","HostId":"1437-test-zone3-test-zonegroup"} 2021-03-11T00:51:50.688 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:50.688 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:50.689 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:51:50.729 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:51:50.730 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:51:50.877 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=76343c7e-8958-4b2a-a007-ca994322980a 2021-03-11T00:51:50.908 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615423909.583891_77.1","last_update":"2021-03-11T00:51:49.583891Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T00:51:50.908 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2021-03-11T00:51:50.909 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:50.909 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:50.910 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:51:50.949 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:51:50.950 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:51:51.136 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423909.583891_77.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:51:49.583891Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:51:51.137 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2021-03-11T00:51:51.137 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T00:51:51.138 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:51.138 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:51.139 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:51:51.221 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:51:51.221 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:51:51.475 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423909.583891_77.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:51:49.583891Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:51:51.476 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T00:51:51.476 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-27 zones={test-zone2, test-zone1} 2021-03-11T00:51:51.498 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:51:51.543 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:51:51.604 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-27 zones={test-zone2, test-zone1} 2021-03-11T00:51:51.604 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-27 zones={test-zone1, test-zone2} 2021-03-11T00:51:51.614 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:51:51.658 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:51:51.721 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-27 zones={test-zone1, test-zone2} 2021-03-11T00:51:51.722 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-28 zones={test-zone2, test-zone1} 2021-03-11T00:51:51.734 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:51:51.783 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:51:51.860 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-28 zones={test-zone2, test-zone1} 2021-03-11T00:51:51.861 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-28 zones={test-zone1, test-zone2} 2021-03-11T00:51:51.872 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:51:51.919 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:51:51.981 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-28 zones={test-zone1, test-zone2} 2021-03-11T00:51:51.982 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-29 zones={test-zone2, test-zone1} 2021-03-11T00:51:51.991 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:51:52.035 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:51:52.092 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-29 zones={test-zone2, test-zone1} 2021-03-11T00:51:52.093 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-29 zones={test-zone1, test-zone2} 2021-03-11T00:51:52.102 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:51:52.151 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:51:52.212 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-29 zones={test-zone1, test-zone2} 2021-03-11T00:51:52.213 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-30 zones={test-zone2, test-zone1} 2021-03-11T00:51:52.223 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:51:52.271 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:51:52.332 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-30 zones={test-zone2, test-zone1} 2021-03-11T00:51:52.333 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-30 zones={test-zone1, test-zone2} 2021-03-11T00:51:52.344 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:51:52.391 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:51:52.450 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-30 zones={test-zone1, test-zone2} 2021-03-11T00:51:52.451 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-31 zones={test-zone2, test-zone1} 2021-03-11T00:51:52.465 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:51:52.511 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:51:52.577 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-31 zones={test-zone2, test-zone1} 2021-03-11T00:51:52.578 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-31 zones={test-zone1, test-zone2} 2021-03-11T00:51:52.587 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:51:52.638 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:51:52.697 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-31 zones={test-zone1, test-zone2} 2021-03-11T00:51:52.698 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-32 zones={test-zone2, test-zone1} 2021-03-11T00:51:52.710 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:51:52.759 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:51:52.824 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-32 zones={test-zone2, test-zone1} 2021-03-11T00:51:52.825 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-32 zones={test-zone1, test-zone2} 2021-03-11T00:51:52.837 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:51:52.887 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:51:52.948 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-32 zones={test-zone1, test-zone2} 2021-03-11T00:51:52.961 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '470410ef-8c66-4339-a004-064d3e2c6c91', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:52.964 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '470410ef-8c66-4339-a004-064d3e2c6c91', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:52.964 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 470410ef-8c66-4339-a004-064d3e2c6c91 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:51:53.004 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:51:53.004 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:51:53.206 INFO:teuthology.orchestra.run.smithi119.stdout:[{"id":"1_1615423453.463276_71.1","section":"user","name":"rgw-multisite-test-user","timestamp":"2021-03-11T00:44:13.463276832Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_PZwaTRfFlIhjjRCXuj8pLzU","ver":1},"status":{"status":"write"}}},{"id":"1_1615423453.469908_73.1","section":"user","name":"rgw-multisite-test-user","timestamp":"2021-03-11T00:44:13.469908140Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_PZwaTRfFlIhjjRCXuj8pLzU","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423619.055431_88.1","section":"bucket.instance","name":"nolznk-10:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.14","timestamp":"2021-03-11T00:46:59.055431642Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_x3avBieUj8FdqeKcsgJp8af","ver":1},"status":{"status":"write"}}},{"id":"1_1615423619.060466_92.1","section":"bucket.instance","name":"nolznk-10:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.14","timestamp":"2021-03-11T00:46:59.060466336Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_x3avBieUj8FdqeKcsgJp8af","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423619.062085_93.1","section":"bucket","name":"nolznk-10","timestamp":"2021-03-11T00:46:59.062085564Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_OXNZTe0e4We59jmLs0rMpT_","ver":1},"status":{"status":"write"}}},{"id":"1_1615423619.066252_94.1","section":"bucket","name":"nolznk-10","timestamp":"2021-03-11T00:46:59.066252978Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_OXNZTe0e4We59jmLs0rMpT_","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423640.022370_95.1","section":"bucket.instance","name":"nolznk-11:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.15","timestamp":"2021-03-11T00:47:20.022370354Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_iWEYGp1FEyAGIrnLCQeNoos","ver":1},"status":{"status":"write"}}},{"id":"1_1615423640.027559_104.1","section":"bucket.instance","name":"nolznk-11:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.15","timestamp":"2021-03-11T00:47:20.027559659Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_iWEYGp1FEyAGIrnLCQeNoos","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423640.029576_105.1","section":"bucket","name":"nolznk-11","timestamp":"2021-03-11T00:47:20.029576858Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zpTXMuPNjT5p6d04tpDlMEN","ver":1},"status":{"status":"write"}}},{"id":"1_1615423640.033640_106.1","section":"bucket","name":"nolznk-11","timestamp":"2021-03-11T00:47:20.033640562Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zpTXMuPNjT5p6d04tpDlMEN","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423640.052872_107.1","section":"bucket.instance","name":"nolznk-12:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.16","timestamp":"2021-03-11T00:47:20.052872922Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_p6o2vK-LPluCv4rY02wZXQP","ver":1},"status":{"status":"write"}}},{"id":"1_1615423640.058960_108.1","section":"bucket.instance","name":"nolznk-12:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.16","timestamp":"2021-03-11T00:47:20.058960731Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_p6o2vK-LPluCv4rY02wZXQP","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423640.060616_109.1","section":"bucket","name":"nolznk-12","timestamp":"2021-03-11T00:47:20.060616083Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_px0OaWGnbTRNA9VFAABcem4","ver":1},"status":{"status":"write"}}},{"id":"1_1615423640.064716_110.1","section":"bucket","name":"nolznk-12","timestamp":"2021-03-11T00:47:20.064716045Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_px0OaWGnbTRNA9VFAABcem4","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423640.092752_111.1","section":"bucket.instance","name":"nolznk-11:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.15","timestamp":"2021-03-11T00:47:20.092752724Z","data":{"read_version":{"tag":"_iWEYGp1FEyAGIrnLCQeNoos","ver":1},"write_version":{"tag":"_iWEYGp1FEyAGIrnLCQeNoos","ver":2},"status":{"status":"write"}}},{"id":"1_1615423640.098733_112.1","section":"bucket.instance","name":"nolznk-11:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.15","timestamp":"2021-03-11T00:47:20.098733298Z","data":{"read_version":{"tag":"_iWEYGp1FEyAGIrnLCQeNoos","ver":1},"write_version":{"tag":"_iWEYGp1FEyAGIrnLCQeNoos","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423640.106848_113.1","section":"bucket.instance","name":"nolznk-12:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.16","timestamp":"2021-03-11T00:47:20.106848462Z","data":{"read_version":{"tag":"_p6o2vK-LPluCv4rY02wZXQP","ver":1},"write_version":{"tag":"_p6o2vK-LPluCv4rY02wZXQP","ver":2},"status":{"status":"write"}}},{"id":"1_1615423640.114087_114.1","section":"bucket.instance","name":"nolznk-12:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.16","timestamp":"2021-03-11T00:47:20.114087357Z","data":{"read_version":{"tag":"_p6o2vK-LPluCv4rY02wZXQP","ver":1},"write_version":{"tag":"_p6o2vK-LPluCv4rY02wZXQP","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423679.987558_115.1","section":"bucket.instance","name":"nolznk-13:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.17","timestamp":"2021-03-11T00:47:59.987558340Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gOLKrQmzpEIXiaID7B7wPti","ver":1},"status":{"status":"write"}}},{"id":"1_1615423679.992413_133.1","section":"bucket.instance","name":"nolznk-13:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.17","timestamp":"2021-03-11T00:47:59.992413924Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gOLKrQmzpEIXiaID7B7wPti","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423679.994531_134.1","section":"bucket","name":"nolznk-13","timestamp":"2021-03-11T00:47:59.994531075Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rbJbxoLyu4bIb9mBPNMUjAn","ver":1},"status":{"status":"write"}}},{"id":"1_1615423679.998997_135.1","section":"bucket","name":"nolznk-13","timestamp":"2021-03-11T00:47:59.998997714Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rbJbxoLyu4bIb9mBPNMUjAn","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423680.004552_136.1","section":"bucket.instance","name":"nolznk-13:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.17","timestamp":"2021-03-11T00:48:00.004552915Z","data":{"read_version":{"tag":"_gOLKrQmzpEIXiaID7B7wPti","ver":1},"write_version":{"tag":"_gOLKrQmzpEIXiaID7B7wPti","ver":2},"status":{"status":"write"}}},{"id":"1_1615423680.010411_137.1","section":"bucket.instance","name":"nolznk-13:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.17","timestamp":"2021-03-11T00:48:00.010411929Z","data":{"read_version":{"tag":"_gOLKrQmzpEIXiaID7B7wPti","ver":1},"write_version":{"tag":"_gOLKrQmzpEIXiaID7B7wPti","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423703.095803_138.1","section":"bucket.instance","name":"nolznk-14:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.18","timestamp":"2021-03-11T00:48:23.095803789Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":1},"status":{"status":"write"}}},{"id":"1_1615423703.100241_149.1","section":"bucket.instance","name":"nolznk-14:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.18","timestamp":"2021-03-11T00:48:23.100241785Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423703.102699_150.1","section":"bucket","name":"nolznk-14","timestamp":"2021-03-11T00:48:23.102699204Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rwmuSTK5NpKSOt_xCqzBBH-","ver":1},"status":{"status":"write"}}},{"id":"1_1615423703.106524_151.1","section":"bucket","name":"nolznk-14","timestamp":"2021-03-11T00:48:23.106524997Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rwmuSTK5NpKSOt_xCqzBBH-","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423716.262443_152.1","section":"bucket.instance","name":"nolznk-14:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.18","timestamp":"2021-03-11T00:48:36.262443475Z","data":{"read_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":1},"write_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":2},"status":{"status":"write"}}},{"id":"1_1615423716.269301_157.1","section":"bucket.instance","name":"nolznk-14:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.18","timestamp":"2021-03-11T00:48:36.269301193Z","data":{"read_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":1},"write_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423729.605376_158.1","section":"bucket.instance","name":"nolznk-14:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.18","timestamp":"2021-03-11T00:48:49.605376112Z","data":{"read_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":2},"write_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":3},"status":{"status":"write"}}},{"id":"1_1615423729.613265_163.1","section":"bucket.instance","name":"nolznk-14:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.18","timestamp":"2021-03-11T00:48:49.613265598Z","data":{"read_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":2},"write_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":3},"status":{"status":"complete"}}},{"id":"1_1615423743.264880_164.1","section":"bucket.instance","name":"nolznk-15:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.19","timestamp":"2021-03-11T00:49:03.264880629Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ObhA4ynS0ZkzOShAmXyPpuK","ver":1},"status":{"status":"write"}}},{"id":"1_1615423743.270054_169.1","section":"bucket.instance","name":"nolznk-15:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.19","timestamp":"2021-03-11T00:49:03.270054481Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ObhA4ynS0ZkzOShAmXyPpuK","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423743.271574_170.1","section":"bucket","name":"nolznk-15","timestamp":"2021-03-11T00:49:03.271574605Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_GsoaEGYqmDbKVeNP79Se_1m","ver":1},"status":{"status":"write"}}},{"id":"1_1615423743.275888_171.1","section":"bucket","name":"nolznk-15","timestamp":"2021-03-11T00:49:03.275888088Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_GsoaEGYqmDbKVeNP79Se_1m","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423743.298858_172.1","section":"bucket.instance","name":"nolznk-16:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.20","timestamp":"2021-03-11T00:49:03.298858602Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_AOlGazAgHoexzcjIgFKH-Mx","ver":1},"status":{"status":"write"}}},{"id":"1_1615423743.303694_173.1","section":"bucket.instance","name":"nolznk-16:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.20","timestamp":"2021-03-11T00:49:03.303694132Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_AOlGazAgHoexzcjIgFKH-Mx","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423743.305295_174.1","section":"bucket","name":"nolznk-16","timestamp":"2021-03-11T00:49:03.305295956Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Ems82ocmFs3LsMnJXhG2suw","ver":1},"status":{"status":"write"}}},{"id":"1_1615423743.309279_175.1","section":"bucket","name":"nolznk-16","timestamp":"2021-03-11T00:49:03.309279431Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Ems82ocmFs3LsMnJXhG2suw","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423743.335935_176.1","section":"bucket.instance","name":"nolznk-15:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.19","timestamp":"2021-03-11T00:49:03.335935935Z","data":{"read_version":{"tag":"_ObhA4ynS0ZkzOShAmXyPpuK","ver":1},"write_version":{"tag":"_ObhA4ynS0ZkzOShAmXyPpuK","ver":2},"status":{"status":"write"}}},{"id":"1_1615423743.341225_177.1","section":"bucket.instance","name":"nolznk-15:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.19","timestamp":"2021-03-11T00:49:03.341225643Z","data":{"read_version":{"tag":"_ObhA4ynS0ZkzOShAmXyPpuK","ver":1},"write_version":{"tag":"_ObhA4ynS0ZkzOShAmXyPpuK","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423743.351429_178.1","section":"bucket.instance","name":"nolznk-16:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.20","timestamp":"2021-03-11T00:49:03.351429534Z","data":{"read_version":{"tag":"_AOlGazAgHoexzcjIgFKH-Mx","ver":1},"write_version":{"tag":"_AOlGazAgHoexzcjIgFKH-Mx","ver":2},"status":{"status":"write"}}},{"id":"1_1615423743.357959_179.1","section":"bucket.instance","name":"nolznk-16:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.20","timestamp":"2021-03-11T00:49:03.357959007Z","data":{"read_version":{"tag":"_AOlGazAgHoexzcjIgFKH-Mx","ver":1},"write_version":{"tag":"_AOlGazAgHoexzcjIgFKH-Mx","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423758.356100_180.1","section":"bucket.instance","name":"nolznk-17:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.21","timestamp":"2021-03-11T00:49:18.356100134Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":1},"status":{"status":"write"}}},{"id":"1_1615423758.360834_188.1","section":"bucket.instance","name":"nolznk-17:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.21","timestamp":"2021-03-11T00:49:18.360834148Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423758.362590_189.1","section":"bucket","name":"nolznk-17","timestamp":"2021-03-11T00:49:18.362590752Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_poFzmuQa6qComI9TTZ1EQU6","ver":1},"status":{"status":"write"}}},{"id":"1_1615423758.366352_190.1","section":"bucket","name":"nolznk-17","timestamp":"2021-03-11T00:49:18.366352196Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_poFzmuQa6qComI9TTZ1EQU6","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423758.387291_191.1","section":"bucket.instance","name":"nolznk-18:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.22","timestamp":"2021-03-11T00:49:18.387291130Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":1},"status":{"status":"write"}}},{"id":"1_1615423758.392936_192.1","section":"bucket.instance","name":"nolznk-18:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.22","timestamp":"2021-03-11T00:49:18.392936703Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423758.394949_193.1","section":"bucket","name":"nolznk-18","timestamp":"2021-03-11T00:49:18.394949665Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mGlZ5EZiAMG_pDJLlo6k4Ao","ver":1},"status":{"status":"write"}}},{"id":"1_1615423758.399367_194.1","section":"bucket","name":"nolznk-18","timestamp":"2021-03-11T00:49:18.399367098Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mGlZ5EZiAMG_pDJLlo6k4Ao","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423758.430274_195.1","section":"bucket.instance","name":"nolznk-17:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.21","timestamp":"2021-03-11T00:49:18.430274655Z","data":{"read_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":1},"write_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":2},"status":{"status":"write"}}},{"id":"1_1615423758.436085_196.1","section":"bucket.instance","name":"nolznk-17:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.21","timestamp":"2021-03-11T00:49:18.436085903Z","data":{"read_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":1},"write_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423758.449253_197.1","section":"bucket.instance","name":"nolznk-17:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.21","timestamp":"2021-03-11T00:49:18.449253989Z","data":{"read_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":2},"write_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":3},"status":{"status":"write"}}},{"id":"1_1615423758.454852_198.1","section":"bucket.instance","name":"nolznk-17:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.21","timestamp":"2021-03-11T00:49:18.454852873Z","data":{"read_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":2},"write_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":3},"status":{"status":"complete"}}},{"id":"1_1615423758.463142_199.1","section":"bucket.instance","name":"nolznk-18:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.22","timestamp":"2021-03-11T00:49:18.463142338Z","data":{"read_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":1},"write_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":2},"status":{"status":"write"}}},{"id":"1_1615423758.469743_200.1","section":"bucket.instance","name":"nolznk-18:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.22","timestamp":"2021-03-11T00:49:18.469743017Z","data":{"read_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":1},"write_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423758.483410_201.1","section":"bucket.instance","name":"nolznk-18:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.22","timestamp":"2021-03-11T00:49:18.483410006Z","data":{"read_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":2},"write_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":3},"status":{"status":"write"}}},{"id":"1_1615423758.490731_202.1","section":"bucket.instance","name":"nolznk-18:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.22","timestamp":"2021-03-11T00:49:18.490731583Z","data":{"read_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":2},"write_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":3},"status":{"status":"complete"}}},{"id":"1_1615423773.188516_203.1","section":"bucket.instance","name":"nolznk-19:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.23","timestamp":"2021-03-11T00:49:33.188516972Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_bfh_EH3cSHh3fV_umZvAZMT","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.193178_211.1","section":"bucket.instance","name":"nolznk-19:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.23","timestamp":"2021-03-11T00:49:33.193178769Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_bfh_EH3cSHh3fV_umZvAZMT","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.194917_212.1","section":"bucket","name":"nolznk-19","timestamp":"2021-03-11T00:49:33.194917296Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_e0eRqOyRBg-S5yTJnjW1lLD","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.199966_213.1","section":"bucket","name":"nolznk-19","timestamp":"2021-03-11T00:49:33.199966925Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_e0eRqOyRBg-S5yTJnjW1lLD","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.219722_214.1","section":"bucket.instance","name":"nolznk-20:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.24","timestamp":"2021-03-11T00:49:33.219722017Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_tROxQwkgsHmcIV2tAtYwLsg","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.225185_215.1","section":"bucket.instance","name":"nolznk-20:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.24","timestamp":"2021-03-11T00:49:33.225185162Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_tROxQwkgsHmcIV2tAtYwLsg","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.226995_216.1","section":"bucket","name":"nolznk-20","timestamp":"2021-03-11T00:49:33.226995917Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_yIdp1ucJJZI_wIl7WJeHezY","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.233185_217.1","section":"bucket","name":"nolznk-20","timestamp":"2021-03-11T00:49:33.233185260Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_yIdp1ucJJZI_wIl7WJeHezY","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.256444_218.1","section":"bucket.instance","name":"nolznk-19:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.23","timestamp":"2021-03-11T00:49:33.256444610Z","data":{"read_version":{"tag":"_bfh_EH3cSHh3fV_umZvAZMT","ver":1},"write_version":{"tag":"_bfh_EH3cSHh3fV_umZvAZMT","ver":2},"status":{"status":"write"}}},{"id":"1_1615423773.262956_219.1","section":"bucket.instance","name":"nolznk-19:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.23","timestamp":"2021-03-11T00:49:33.262956244Z","data":{"read_version":{"tag":"_bfh_EH3cSHh3fV_umZvAZMT","ver":1},"write_version":{"tag":"_bfh_EH3cSHh3fV_umZvAZMT","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423773.314286_220.1","section":"bucket.instance","name":"nolznk-20:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.24","timestamp":"2021-03-11T00:49:33.314286791Z","data":{"read_version":{"tag":"_tROxQwkgsHmcIV2tAtYwLsg","ver":1},"write_version":{"tag":"_tROxQwkgsHmcIV2tAtYwLsg","ver":2},"status":{"status":"write"}}},{"id":"1_1615423773.322365_221.1","section":"bucket.instance","name":"nolznk-20:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.24","timestamp":"2021-03-11T00:49:33.322365087Z","data":{"read_version":{"tag":"_tROxQwkgsHmcIV2tAtYwLsg","ver":1},"write_version":{"tag":"_tROxQwkgsHmcIV2tAtYwLsg","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423773.388830_222.1","section":"bucket.instance","name":"nolznk-21:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.25","timestamp":"2021-03-11T00:49:33.388830385Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DtCBNb6yWQJ9bljYZUL-Tpd","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.393488_223.1","section":"bucket.instance","name":"nolznk-21:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.25","timestamp":"2021-03-11T00:49:33.393488957Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DtCBNb6yWQJ9bljYZUL-Tpd","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.394906_224.1","section":"bucket","name":"nolznk-21","timestamp":"2021-03-11T00:49:33.394906992Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_V6Lv5bVW3vMXiLEcXPgSdoy","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.398672_225.1","section":"bucket","name":"nolznk-21","timestamp":"2021-03-11T00:49:33.398672730Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_V6Lv5bVW3vMXiLEcXPgSdoy","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.413829_226.1","section":"bucket.instance","name":"nolznk-22:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.26","timestamp":"2021-03-11T00:49:33.413829448Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VqPhf4iuPtHx-nZ4zTRaSwu","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.418512_227.1","section":"bucket.instance","name":"nolznk-22:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.26","timestamp":"2021-03-11T00:49:33.418512596Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VqPhf4iuPtHx-nZ4zTRaSwu","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.420358_228.1","section":"bucket","name":"nolznk-22","timestamp":"2021-03-11T00:49:33.420358132Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_TQxPB4z8uyCZjrmcrjABQiV","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.425060_229.1","section":"bucket","name":"nolznk-22","timestamp":"2021-03-11T00:49:33.425060654Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_TQxPB4z8uyCZjrmcrjABQiV","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.450023_230.1","section":"bucket.instance","name":"nolznk-21:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.25","timestamp":"2021-03-11T00:49:33.450023398Z","data":{"read_version":{"tag":"_DtCBNb6yWQJ9bljYZUL-Tpd","ver":1},"write_version":{"tag":"_DtCBNb6yWQJ9bljYZUL-Tpd","ver":2},"status":{"status":"write"}}},{"id":"1_1615423773.456451_231.1","section":"bucket.instance","name":"nolznk-21:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.25","timestamp":"2021-03-11T00:49:33.456451795Z","data":{"read_version":{"tag":"_DtCBNb6yWQJ9bljYZUL-Tpd","ver":1},"write_version":{"tag":"_DtCBNb6yWQJ9bljYZUL-Tpd","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423773.511182_232.1","section":"bucket.instance","name":"nolznk-22:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.26","timestamp":"2021-03-11T00:49:33.511182120Z","data":{"read_version":{"tag":"_VqPhf4iuPtHx-nZ4zTRaSwu","ver":1},"write_version":{"tag":"_VqPhf4iuPtHx-nZ4zTRaSwu","ver":2},"status":{"status":"write"}}},{"id":"1_1615423773.519683_233.1","section":"bucket.instance","name":"nolznk-22:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.26","timestamp":"2021-03-11T00:49:33.519683540Z","data":{"read_version":{"tag":"_VqPhf4iuPtHx-nZ4zTRaSwu","ver":1},"write_version":{"tag":"_VqPhf4iuPtHx-nZ4zTRaSwu","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423773.595382_234.1","section":"bucket.instance","name":"nolznk-23:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.27","timestamp":"2021-03-11T00:49:33.595382068Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Kd3YMYl8eXvrLGACqaoWSFq","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.600025_235.1","section":"bucket.instance","name":"nolznk-23:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.27","timestamp":"2021-03-11T00:49:33.600025216Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Kd3YMYl8eXvrLGACqaoWSFq","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.601433_236.1","section":"bucket","name":"nolznk-23","timestamp":"2021-03-11T00:49:33.601433770Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_3itXKzsUPhzhes2xUErh0Qx","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.604979_237.1","section":"bucket","name":"nolznk-23","timestamp":"2021-03-11T00:49:33.604979959Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_3itXKzsUPhzhes2xUErh0Qx","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.623668_238.1","section":"bucket.instance","name":"nolznk-24:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.28","timestamp":"2021-03-11T00:49:33.623668196Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DEVpH-WkgJI7HpB3UAQY0hj","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.629418_239.1","section":"bucket.instance","name":"nolznk-24:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.28","timestamp":"2021-03-11T00:49:33.629418819Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DEVpH-WkgJI7HpB3UAQY0hj","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.631233_240.1","section":"bucket","name":"nolznk-24","timestamp":"2021-03-11T00:49:33.631233214Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zWzxEZawxCPuZRL3VFOI87f","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.638186_241.1","section":"bucket","name":"nolznk-24","timestamp":"2021-03-11T00:49:33.638186236Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zWzxEZawxCPuZRL3VFOI87f","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.661473_242.1","section":"bucket.instance","name":"nolznk-23:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.27","timestamp":"2021-03-11T00:49:33.661473290Z","data":{"read_version":{"tag":"_Kd3YMYl8eXvrLGACqaoWSFq","ver":1},"write_version":{"tag":"_Kd3YMYl8eXvrLGACqaoWSFq","ver":2},"status":{"status":"write"}}},{"id":"1_1615423773.666810_243.1","section":"bucket.instance","name":"nolznk-23:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.27","timestamp":"2021-03-11T00:49:33.666810087Z","data":{"read_version":{"tag":"_Kd3YMYl8eXvrLGACqaoWSFq","ver":1},"write_version":{"tag":"_Kd3YMYl8eXvrLGACqaoWSFq","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423774.723261_244.1","section":"bucket.instance","name":"nolznk-24:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.28","timestamp":"2021-03-11T00:49:34.723261077Z","data":{"read_version":{"tag":"_DEVpH-WkgJI7HpB3UAQY0hj","ver":1},"write_version":{"tag":"_DEVpH-WkgJI7HpB3UAQY0hj","ver":2},"status":{"status":"write"}}},{"id":"1_1615423774.732056_245.1","section":"bucket.instance","name":"nolznk-24:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.28","timestamp":"2021-03-11T00:49:34.732056339Z","data":{"read_version":{"tag":"_DEVpH-WkgJI7HpB3UAQY0hj","ver":1},"write_version":{"tag":"_DEVpH-WkgJI7HpB3UAQY0hj","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423774.807833_246.1","section":"bucket.instance","name":"nolznk-25:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.29","timestamp":"2021-03-11T00:49:34.807833043Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_K4b3tWotdt3YxAzVMCBXaXJ","ver":1},"status":{"status":"write"}}},{"id":"1_1615423774.812726_247.1","section":"bucket.instance","name":"nolznk-25:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.29","timestamp":"2021-03-11T00:49:34.812726730Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_K4b3tWotdt3YxAzVMCBXaXJ","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423774.814456_248.1","section":"bucket","name":"nolznk-25","timestamp":"2021-03-11T00:49:34.814456608Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_CGAgOQYApDaSMClQOdo1ewR","ver":1},"status":{"status":"write"}}},{"id":"1_1615423774.819187_249.1","section":"bucket","name":"nolznk-25","timestamp":"2021-03-11T00:49:34.819187613Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_CGAgOQYApDaSMClQOdo1ewR","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423774.837944_250.1","section":"bucket.instance","name":"nolznk-26:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.30","timestamp":"2021-03-11T00:49:34.837944290Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_E3IIKzX9z59ojuvBjbsF3pq","ver":1},"status":{"status":"write"}}},{"id":"1_1615423774.842943_251.1","section":"bucket.instance","name":"nolznk-26:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.30","timestamp":"2021-03-11T00:49:34.842943745Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_E3IIKzX9z59ojuvBjbsF3pq","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423774.844805_252.1","section":"bucket","name":"nolznk-26","timestamp":"2021-03-11T00:49:34.844805420Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_e5EhGvIChq8be4lQdVIoFrQ","ver":1},"status":{"status":"write"}}},{"id":"1_1615423774.849040_253.1","section":"bucket","name":"nolznk-26","timestamp":"2021-03-11T00:49:34.849040958Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_e5EhGvIChq8be4lQdVIoFrQ","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423775.964068_256.1","section":"bucket.instance","name":"nolznk-27:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.31","timestamp":"2021-03-11T00:49:35.964068040Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Rzrci_t2792wGHI9TE4HTRC","ver":1},"status":{"status":"write"}}},{"id":"1_1615423775.969096_257.1","section":"bucket.instance","name":"nolznk-27:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.31","timestamp":"2021-03-11T00:49:35.969096591Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Rzrci_t2792wGHI9TE4HTRC","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423775.970963_258.1","section":"bucket","name":"nolznk-27","timestamp":"2021-03-11T00:49:35.970963524Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_O-sq5_XFIRPaIH9vjf7rIT7","ver":1},"status":{"status":"write"}}},{"id":"1_1615423775.975158_259.1","section":"bucket","name":"nolznk-27","timestamp":"2021-03-11T00:49:35.975158527Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_O-sq5_XFIRPaIH9vjf7rIT7","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423775.995326_260.1","section":"bucket.instance","name":"nolznk-28:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.32","timestamp":"2021-03-11T00:49:35.995326177Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_GdEznOSqpGleDhPhmiqDwDN","ver":1},"status":{"status":"write"}}},{"id":"1_1615423776.000972_261.1","section":"bucket.instance","name":"nolznk-28:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.32","timestamp":"2021-03-11T00:49:36.000972074Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_GdEznOSqpGleDhPhmiqDwDN","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423776.002878_262.1","section":"bucket","name":"nolznk-28","timestamp":"2021-03-11T00:49:36.002878628Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ZHx0nQHYkA_1K9-aju8drea","ver":1},"status":{"status":"write"}}},{"id":"1_1615423776.006942_263.1","section":"bucket","name":"nolznk-28","timestamp":"2021-03-11T00:49:36.006942179Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ZHx0nQHYkA_1K9-aju8drea","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423573.779224_33.1","section":"bucket.instance","name":"nolznk-1:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.1","timestamp":"2021-03-11T00:46:13.779224658Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2t_xvaNhQOSvG3UmDNb-Gmd","ver":1},"status":{"status":"write"}}},{"id":"1_1615423573.785608_34.1","section":"bucket.instance","name":"nolznk-1:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.1","timestamp":"2021-03-11T00:46:13.785608470Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2t_xvaNhQOSvG3UmDNb-Gmd","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423573.787427_35.1","section":"bucket","name":"nolznk-1","timestamp":"2021-03-11T00:46:13.787427525Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NBtj2vMajUaNF8VeKLq1iYO","ver":1},"status":{"status":"write"}}},{"id":"1_1615423573.793388_36.1","section":"bucket","name":"nolznk-1","timestamp":"2021-03-11T00:46:13.793388906Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NBtj2vMajUaNF8VeKLq1iYO","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423573.819219_37.1","section":"bucket.instance","name":"nolznk-2:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.2","timestamp":"2021-03-11T00:46:13.819219101Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gQrL8atL6exiI5rQgMzmgvQ","ver":1},"status":{"status":"write"}}},{"id":"1_1615423573.824837_38.1","section":"bucket.instance","name":"nolznk-2:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.2","timestamp":"2021-03-11T00:46:13.824837055Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gQrL8atL6exiI5rQgMzmgvQ","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423573.826464_39.1","section":"bucket","name":"nolznk-2","timestamp":"2021-03-11T00:46:13.826464055Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_C6tTZv7dm-BiN39XejTafFV","ver":1},"status":{"status":"write"}}},{"id":"1_1615423573.832639_40.1","section":"bucket","name":"nolznk-2","timestamp":"2021-03-11T00:46:13.832639639Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_C6tTZv7dm-BiN39XejTafFV","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423583.648205_41.1","section":"bucket.instance","name":"nolznk-3:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.3","timestamp":"2021-03-11T00:46:23.648205263Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FGN9jBB7YE_ghkHcU1I3LNt","ver":1},"status":{"status":"write"}}},{"id":"1_1615423583.653383_42.1","section":"bucket.instance","name":"nolznk-3:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.3","timestamp":"2021-03-11T00:46:23.653383372Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FGN9jBB7YE_ghkHcU1I3LNt","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423583.655101_43.1","section":"bucket","name":"nolznk-3","timestamp":"2021-03-11T00:46:23.655101249Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_70McMugXBxcL_bL1Cmifz6L","ver":1},"status":{"status":"write"}}},{"id":"1_1615423583.659946_44.1","section":"bucket","name":"nolznk-3","timestamp":"2021-03-11T00:46:23.659946393Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_70McMugXBxcL_bL1Cmifz6L","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423583.679743_45.1","section":"bucket.instance","name":"nolznk-4:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.4","timestamp":"2021-03-11T00:46:23.679743936Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Gr35hKIaC-zpMhxqBvTX4Nu","ver":1},"status":{"status":"write"}}},{"id":"1_1615423583.686290_46.1","section":"bucket.instance","name":"nolznk-4:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.4","timestamp":"2021-03-11T00:46:23.686290426Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Gr35hKIaC-zpMhxqBvTX4Nu","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423583.688328_47.1","section":"bucket","name":"nolznk-4","timestamp":"2021-03-11T00:46:23.688328527Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LCs2kQqpSnxfkqp77rlA4Uh","ver":1},"status":{"status":"write"}}},{"id":"1_1615423583.694980_48.1","section":"bucket","name":"nolznk-4","timestamp":"2021-03-11T00:46:23.694980993Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LCs2kQqpSnxfkqp77rlA4Uh","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423589.811650_51.1","section":"bucket.instance","name":"nolznk-3:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.5","timestamp":"2021-03-11T00:46:29.811650218Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4UUEYyix76dlhi5gQ96jeZO","ver":1},"status":{"status":"write"}}},{"id":"1_1615423589.819550_52.1","section":"bucket.instance","name":"nolznk-3:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.5","timestamp":"2021-03-11T00:46:29.819550644Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4UUEYyix76dlhi5gQ96jeZO","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423589.821577_53.1","section":"bucket","name":"nolznk-3","timestamp":"2021-03-11T00:46:29.821577182Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_nNni5geDzI_cILaAduVvArT","ver":1},"status":{"status":"write"}}},{"id":"1_1615423589.826156_54.1","section":"bucket","name":"nolznk-3","timestamp":"2021-03-11T00:46:29.826156290Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_nNni5geDzI_cILaAduVvArT","ver":1},"status":{"status":"abort"}}},{"id":"1_1615423589.855852_55.1","section":"bucket.instance","name":"nolznk-4:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.6","timestamp":"2021-03-11T00:46:29.855852336Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_uL4uyLLlOHfXnXpokAAu-15","ver":1},"status":{"status":"write"}}},{"id":"1_1615423589.860662_56.1","section":"bucket.instance","name":"nolznk-4:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.6","timestamp":"2021-03-11T00:46:29.860662818Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_uL4uyLLlOHfXnXpokAAu-15","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423589.862946_57.1","section":"bucket","name":"nolznk-4","timestamp":"2021-03-11T00:46:29.862946094Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_C3MbfKZdBWsojefkVvMyhqL","ver":1},"status":{"status":"write"}}},{"id":"1_1615423589.865927_58.1","section":"bucket","name":"nolznk-4","timestamp":"2021-03-11T00:46:29.865927078Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_C3MbfKZdBWsojefkVvMyhqL","ver":1},"status":{"status":"abort"}}},{"id":"1_1615423589.892960_60.1","section":"bucket.instance","name":"nolznk-3:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.7","timestamp":"2021-03-11T00:46:29.892960158Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JW05anS1Np_EfOSPeKyf_yM","ver":1},"status":{"status":"write"}}},{"id":"1_1615423589.899828_61.1","section":"bucket.instance","name":"nolznk-3:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.7","timestamp":"2021-03-11T00:46:29.899828307Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JW05anS1Np_EfOSPeKyf_yM","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423589.901666_62.1","section":"bucket","name":"nolznk-3","timestamp":"2021-03-11T00:46:29.901666754Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_5-d9Q2sYdsRMLXMOsdZg1E-","ver":1},"status":{"status":"write"}}},{"id":"1_1615423589.906825_63.1","section":"bucket","name":"nolznk-3","timestamp":"2021-03-11T00:46:29.906825704Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_5-d9Q2sYdsRMLXMOsdZg1E-","ver":1},"status":{"status":"abort"}}},{"id":"1_1615423589.954490_64.1","section":"bucket.instance","name":"nolznk-4:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.8","timestamp":"2021-03-11T00:46:29.954490622Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_EoLm3bRg_s7LjIXIXwQ2bGl","ver":1},"status":{"status":"write"}}},{"id":"1_1615423589.961897_65.1","section":"bucket.instance","name":"nolznk-4:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.8","timestamp":"2021-03-11T00:46:29.961897362Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_EoLm3bRg_s7LjIXIXwQ2bGl","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423589.963929_66.1","section":"bucket","name":"nolznk-4","timestamp":"2021-03-11T00:46:29.963929759Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_m6SQ98L0z2c9tpG7mwZpVCr","ver":1},"status":{"status":"write"}}},{"id":"1_1615423589.969156_67.1","section":"bucket","name":"nolznk-4","timestamp":"2021-03-11T00:46:29.969156315Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_m6SQ98L0z2c9tpG7mwZpVCr","ver":1},"status":{"status":"abort"}}},{"id":"1_1615423596.160393_71.1","section":"bucket.instance","name":"nolznk-5:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.9","timestamp":"2021-03-11T00:46:36.160393574Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_abeZIu_LDX-6fyUsj8leiUH","ver":1},"status":{"status":"write"}}},{"id":"1_1615423596.166034_72.1","section":"bucket.instance","name":"nolznk-5:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.9","timestamp":"2021-03-11T00:46:36.166034845Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_abeZIu_LDX-6fyUsj8leiUH","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423596.168075_73.1","section":"bucket","name":"nolznk-5","timestamp":"2021-03-11T00:46:36.168075697Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_OPM8B0wzAs7xqCYsHKqP2IP","ver":1},"status":{"status":"write"}}},{"id":"1_1615423596.172637_74.1","section":"bucket","name":"nolznk-5","timestamp":"2021-03-11T00:46:36.172637219Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_OPM8B0wzAs7xqCYsHKqP2IP","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423596.188375_75.1","section":"bucket.instance","name":"nolznk-6:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.10","timestamp":"2021-03-11T00:46:36.188375006Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_CKdgBxyJNwgGTATsCGLE-xq","ver":1},"status":{"status":"write"}}},{"id":"1_1615423596.193886_76.1","section":"bucket.instance","name":"nolznk-6:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.10","timestamp":"2021-03-11T00:46:36.193886239Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_CKdgBxyJNwgGTATsCGLE-xq","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423596.195668_77.1","section":"bucket","name":"nolznk-6","timestamp":"2021-03-11T00:46:36.195668270Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_t15sglXCLoz3bXTGemUXlX_","ver":1},"status":{"status":"write"}}},{"id":"1_1615423596.201627_78.1","section":"bucket","name":"nolznk-6","timestamp":"2021-03-11T00:46:36.201627546Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_t15sglXCLoz3bXTGemUXlX_","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423602.311057_79.1","section":"bucket","name":"nolznk-5","timestamp":"2021-03-11T00:46:42.311057643Z","data":{"read_version":{"tag":"_OPM8B0wzAs7xqCYsHKqP2IP","ver":1},"write_version":{"tag":"_OPM8B0wzAs7xqCYsHKqP2IP","ver":2},"status":{"status":"remove"}}},{"id":"1_1615423602.316644_80.1","section":"bucket","name":"nolznk-5","timestamp":"2021-03-11T00:46:42.316644434Z","data":{"read_version":{"tag":"_OPM8B0wzAs7xqCYsHKqP2IP","ver":1},"write_version":{"tag":"_OPM8B0wzAs7xqCYsHKqP2IP","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423602.356260_81.1","section":"bucket","name":"nolznk-6","timestamp":"2021-03-11T00:46:42.356260413Z","data":{"read_version":{"tag":"_t15sglXCLoz3bXTGemUXlX_","ver":1},"write_version":{"tag":"_t15sglXCLoz3bXTGemUXlX_","ver":2},"status":{"status":"remove"}}},{"id":"1_1615423602.361914_82.1","section":"bucket","name":"nolznk-6","timestamp":"2021-03-11T00:46:42.361914200Z","data":{"read_version":{"tag":"_t15sglXCLoz3bXTGemUXlX_","ver":1},"write_version":{"tag":"_t15sglXCLoz3bXTGemUXlX_","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423608.500814_83.1","section":"bucket.instance","name":"nolznk-7:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.11","timestamp":"2021-03-11T00:46:48.500814832Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_5q5LTZ7iRVPMyiqmZBWgMZZ","ver":1},"status":{"status":"write"}}},{"id":"1_1615423608.506444_84.1","section":"bucket.instance","name":"nolznk-7:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.11","timestamp":"2021-03-11T00:46:48.506444397Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_5q5LTZ7iRVPMyiqmZBWgMZZ","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423608.507991_85.1","section":"bucket","name":"nolznk-7","timestamp":"2021-03-11T00:46:48.507991136Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Fzog3gQiW8a5jzyMWvnY65O","ver":1},"status":{"status":"write"}}},{"id":"1_1615423608.511544_86.1","section":"bucket","name":"nolznk-7","timestamp":"2021-03-11T00:46:48.511544979Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Fzog3gQiW8a5jzyMWvnY65O","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423608.527511_87.1","section":"bucket.instance","name":"nolznk-8:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.12","timestamp":"2021-03-11T00:46:48.527511620Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_yLo2o4q2UdLAGI0c-nBeaK-","ver":1},"status":{"status":"write"}}},{"id":"1_1615423608.532152_88.1","section":"bucket.instance","name":"nolznk-8:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.12","timestamp":"2021-03-11T00:46:48.532152451Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_yLo2o4q2UdLAGI0c-nBeaK-","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423608.533931_89.1","section":"bucket","name":"nolznk-8","timestamp":"2021-03-11T00:46:48.533931862Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_tCC2Km0qZUmiqiViuLUsIA0","ver":1},"status":{"status":"write"}}},{"id":"1_1615423608.538319_90.1","section":"bucket","name":"nolznk-8","timestamp":"2021-03-11T00:46:48.538319128Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_tCC2Km0qZUmiqiViuLUsIA0","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423619.023584_98.1","section":"bucket.instance","name":"nolznk-9:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.13","timestamp":"2021-03-11T00:46:59.023584771Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_tAf18T7A-WGzT8ffKm9GGgu","ver":1},"status":{"status":"write"}}},{"id":"1_1615423619.028138_99.1","section":"bucket.instance","name":"nolznk-9:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.13","timestamp":"2021-03-11T00:46:59.028138621Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_tAf18T7A-WGzT8ffKm9GGgu","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423619.029573_100.1","section":"bucket","name":"nolznk-9","timestamp":"2021-03-11T00:46:59.029573417Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Yu95E0IeKnKtPiXbPggWQ80","ver":1},"status":{"status":"write"}}},{"id":"1_1615423619.033309_101.1","section":"bucket","name":"nolznk-9","timestamp":"2021-03-11T00:46:59.033309932Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Yu95E0IeKnKtPiXbPggWQ80","ver":1},"status":{"status":"complete"}}}] 2021-03-11T00:51:53.209 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '470410ef-8c66-4339-a004-064d3e2c6c91', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:53.210 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '470410ef-8c66-4339-a004-064d3e2c6c91', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:53.211 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 470410ef-8c66-4339-a004-064d3e2c6c91 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:51:53.251 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:51:53.251 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:51:53.439 INFO:teuthology.orchestra.run.smithi124.stdout:[{"id":"1_1615423453.463276_71.1","section":"user","name":"rgw-multisite-test-user","timestamp":"2021-03-11T00:44:13.463276832Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_PZwaTRfFlIhjjRCXuj8pLzU","ver":1},"status":{"status":"write"}}},{"id":"1_1615423453.469908_73.1","section":"user","name":"rgw-multisite-test-user","timestamp":"2021-03-11T00:44:13.469908140Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_PZwaTRfFlIhjjRCXuj8pLzU","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423619.055431_88.1","section":"bucket.instance","name":"nolznk-10:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.14","timestamp":"2021-03-11T00:46:59.055431642Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_x3avBieUj8FdqeKcsgJp8af","ver":1},"status":{"status":"write"}}},{"id":"1_1615423619.060466_92.1","section":"bucket.instance","name":"nolznk-10:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.14","timestamp":"2021-03-11T00:46:59.060466336Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_x3avBieUj8FdqeKcsgJp8af","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423619.062085_93.1","section":"bucket","name":"nolznk-10","timestamp":"2021-03-11T00:46:59.062085564Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_OXNZTe0e4We59jmLs0rMpT_","ver":1},"status":{"status":"write"}}},{"id":"1_1615423619.066252_94.1","section":"bucket","name":"nolznk-10","timestamp":"2021-03-11T00:46:59.066252978Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_OXNZTe0e4We59jmLs0rMpT_","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423640.022370_95.1","section":"bucket.instance","name":"nolznk-11:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.15","timestamp":"2021-03-11T00:47:20.022370354Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_iWEYGp1FEyAGIrnLCQeNoos","ver":1},"status":{"status":"write"}}},{"id":"1_1615423640.027559_104.1","section":"bucket.instance","name":"nolznk-11:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.15","timestamp":"2021-03-11T00:47:20.027559659Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_iWEYGp1FEyAGIrnLCQeNoos","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423640.029576_105.1","section":"bucket","name":"nolznk-11","timestamp":"2021-03-11T00:47:20.029576858Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zpTXMuPNjT5p6d04tpDlMEN","ver":1},"status":{"status":"write"}}},{"id":"1_1615423640.033640_106.1","section":"bucket","name":"nolznk-11","timestamp":"2021-03-11T00:47:20.033640562Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zpTXMuPNjT5p6d04tpDlMEN","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423640.052872_107.1","section":"bucket.instance","name":"nolznk-12:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.16","timestamp":"2021-03-11T00:47:20.052872922Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_p6o2vK-LPluCv4rY02wZXQP","ver":1},"status":{"status":"write"}}},{"id":"1_1615423640.058960_108.1","section":"bucket.instance","name":"nolznk-12:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.16","timestamp":"2021-03-11T00:47:20.058960731Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_p6o2vK-LPluCv4rY02wZXQP","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423640.060616_109.1","section":"bucket","name":"nolznk-12","timestamp":"2021-03-11T00:47:20.060616083Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_px0OaWGnbTRNA9VFAABcem4","ver":1},"status":{"status":"write"}}},{"id":"1_1615423640.064716_110.1","section":"bucket","name":"nolznk-12","timestamp":"2021-03-11T00:47:20.064716045Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_px0OaWGnbTRNA9VFAABcem4","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423640.092752_111.1","section":"bucket.instance","name":"nolznk-11:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.15","timestamp":"2021-03-11T00:47:20.092752724Z","data":{"read_version":{"tag":"_iWEYGp1FEyAGIrnLCQeNoos","ver":1},"write_version":{"tag":"_iWEYGp1FEyAGIrnLCQeNoos","ver":2},"status":{"status":"write"}}},{"id":"1_1615423640.098733_112.1","section":"bucket.instance","name":"nolznk-11:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.15","timestamp":"2021-03-11T00:47:20.098733298Z","data":{"read_version":{"tag":"_iWEYGp1FEyAGIrnLCQeNoos","ver":1},"write_version":{"tag":"_iWEYGp1FEyAGIrnLCQeNoos","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423640.106848_113.1","section":"bucket.instance","name":"nolznk-12:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.16","timestamp":"2021-03-11T00:47:20.106848462Z","data":{"read_version":{"tag":"_p6o2vK-LPluCv4rY02wZXQP","ver":1},"write_version":{"tag":"_p6o2vK-LPluCv4rY02wZXQP","ver":2},"status":{"status":"write"}}},{"id":"1_1615423640.114087_114.1","section":"bucket.instance","name":"nolznk-12:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.16","timestamp":"2021-03-11T00:47:20.114087357Z","data":{"read_version":{"tag":"_p6o2vK-LPluCv4rY02wZXQP","ver":1},"write_version":{"tag":"_p6o2vK-LPluCv4rY02wZXQP","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423679.987558_115.1","section":"bucket.instance","name":"nolznk-13:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.17","timestamp":"2021-03-11T00:47:59.987558340Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gOLKrQmzpEIXiaID7B7wPti","ver":1},"status":{"status":"write"}}},{"id":"1_1615423679.992413_133.1","section":"bucket.instance","name":"nolznk-13:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.17","timestamp":"2021-03-11T00:47:59.992413924Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gOLKrQmzpEIXiaID7B7wPti","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423679.994531_134.1","section":"bucket","name":"nolznk-13","timestamp":"2021-03-11T00:47:59.994531075Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rbJbxoLyu4bIb9mBPNMUjAn","ver":1},"status":{"status":"write"}}},{"id":"1_1615423679.998997_135.1","section":"bucket","name":"nolznk-13","timestamp":"2021-03-11T00:47:59.998997714Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rbJbxoLyu4bIb9mBPNMUjAn","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423680.004552_136.1","section":"bucket.instance","name":"nolznk-13:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.17","timestamp":"2021-03-11T00:48:00.004552915Z","data":{"read_version":{"tag":"_gOLKrQmzpEIXiaID7B7wPti","ver":1},"write_version":{"tag":"_gOLKrQmzpEIXiaID7B7wPti","ver":2},"status":{"status":"write"}}},{"id":"1_1615423680.010411_137.1","section":"bucket.instance","name":"nolznk-13:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.17","timestamp":"2021-03-11T00:48:00.010411929Z","data":{"read_version":{"tag":"_gOLKrQmzpEIXiaID7B7wPti","ver":1},"write_version":{"tag":"_gOLKrQmzpEIXiaID7B7wPti","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423703.095803_138.1","section":"bucket.instance","name":"nolznk-14:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.18","timestamp":"2021-03-11T00:48:23.095803789Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":1},"status":{"status":"write"}}},{"id":"1_1615423703.100241_149.1","section":"bucket.instance","name":"nolznk-14:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.18","timestamp":"2021-03-11T00:48:23.100241785Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423703.102699_150.1","section":"bucket","name":"nolznk-14","timestamp":"2021-03-11T00:48:23.102699204Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rwmuSTK5NpKSOt_xCqzBBH-","ver":1},"status":{"status":"write"}}},{"id":"1_1615423703.106524_151.1","section":"bucket","name":"nolznk-14","timestamp":"2021-03-11T00:48:23.106524997Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rwmuSTK5NpKSOt_xCqzBBH-","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423716.262443_152.1","section":"bucket.instance","name":"nolznk-14:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.18","timestamp":"2021-03-11T00:48:36.262443475Z","data":{"read_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":1},"write_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":2},"status":{"status":"write"}}},{"id":"1_1615423716.269301_157.1","section":"bucket.instance","name":"nolznk-14:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.18","timestamp":"2021-03-11T00:48:36.269301193Z","data":{"read_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":1},"write_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423729.605376_158.1","section":"bucket.instance","name":"nolznk-14:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.18","timestamp":"2021-03-11T00:48:49.605376112Z","data":{"read_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":2},"write_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":3},"status":{"status":"write"}}},{"id":"1_1615423729.613265_163.1","section":"bucket.instance","name":"nolznk-14:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.18","timestamp":"2021-03-11T00:48:49.613265598Z","data":{"read_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":2},"write_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":3},"status":{"status":"complete"}}},{"id":"1_1615423743.264880_164.1","section":"bucket.instance","name":"nolznk-15:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.19","timestamp":"2021-03-11T00:49:03.264880629Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ObhA4ynS0ZkzOShAmXyPpuK","ver":1},"status":{"status":"write"}}},{"id":"1_1615423743.270054_169.1","section":"bucket.instance","name":"nolznk-15:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.19","timestamp":"2021-03-11T00:49:03.270054481Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ObhA4ynS0ZkzOShAmXyPpuK","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423743.271574_170.1","section":"bucket","name":"nolznk-15","timestamp":"2021-03-11T00:49:03.271574605Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_GsoaEGYqmDbKVeNP79Se_1m","ver":1},"status":{"status":"write"}}},{"id":"1_1615423743.275888_171.1","section":"bucket","name":"nolznk-15","timestamp":"2021-03-11T00:49:03.275888088Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_GsoaEGYqmDbKVeNP79Se_1m","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423743.298858_172.1","section":"bucket.instance","name":"nolznk-16:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.20","timestamp":"2021-03-11T00:49:03.298858602Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_AOlGazAgHoexzcjIgFKH-Mx","ver":1},"status":{"status":"write"}}},{"id":"1_1615423743.303694_173.1","section":"bucket.instance","name":"nolznk-16:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.20","timestamp":"2021-03-11T00:49:03.303694132Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_AOlGazAgHoexzcjIgFKH-Mx","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423743.305295_174.1","section":"bucket","name":"nolznk-16","timestamp":"2021-03-11T00:49:03.305295956Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Ems82ocmFs3LsMnJXhG2suw","ver":1},"status":{"status":"write"}}},{"id":"1_1615423743.309279_175.1","section":"bucket","name":"nolznk-16","timestamp":"2021-03-11T00:49:03.309279431Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Ems82ocmFs3LsMnJXhG2suw","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423743.335935_176.1","section":"bucket.instance","name":"nolznk-15:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.19","timestamp":"2021-03-11T00:49:03.335935935Z","data":{"read_version":{"tag":"_ObhA4ynS0ZkzOShAmXyPpuK","ver":1},"write_version":{"tag":"_ObhA4ynS0ZkzOShAmXyPpuK","ver":2},"status":{"status":"write"}}},{"id":"1_1615423743.341225_177.1","section":"bucket.instance","name":"nolznk-15:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.19","timestamp":"2021-03-11T00:49:03.341225643Z","data":{"read_version":{"tag":"_ObhA4ynS0ZkzOShAmXyPpuK","ver":1},"write_version":{"tag":"_ObhA4ynS0ZkzOShAmXyPpuK","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423743.351429_178.1","section":"bucket.instance","name":"nolznk-16:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.20","timestamp":"2021-03-11T00:49:03.351429534Z","data":{"read_version":{"tag":"_AOlGazAgHoexzcjIgFKH-Mx","ver":1},"write_version":{"tag":"_AOlGazAgHoexzcjIgFKH-Mx","ver":2},"status":{"status":"write"}}},{"id":"1_1615423743.357959_179.1","section":"bucket.instance","name":"nolznk-16:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.20","timestamp":"2021-03-11T00:49:03.357959007Z","data":{"read_version":{"tag":"_AOlGazAgHoexzcjIgFKH-Mx","ver":1},"write_version":{"tag":"_AOlGazAgHoexzcjIgFKH-Mx","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423758.356100_180.1","section":"bucket.instance","name":"nolznk-17:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.21","timestamp":"2021-03-11T00:49:18.356100134Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":1},"status":{"status":"write"}}},{"id":"1_1615423758.360834_188.1","section":"bucket.instance","name":"nolznk-17:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.21","timestamp":"2021-03-11T00:49:18.360834148Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423758.362590_189.1","section":"bucket","name":"nolznk-17","timestamp":"2021-03-11T00:49:18.362590752Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_poFzmuQa6qComI9TTZ1EQU6","ver":1},"status":{"status":"write"}}},{"id":"1_1615423758.366352_190.1","section":"bucket","name":"nolznk-17","timestamp":"2021-03-11T00:49:18.366352196Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_poFzmuQa6qComI9TTZ1EQU6","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423758.387291_191.1","section":"bucket.instance","name":"nolznk-18:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.22","timestamp":"2021-03-11T00:49:18.387291130Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":1},"status":{"status":"write"}}},{"id":"1_1615423758.392936_192.1","section":"bucket.instance","name":"nolznk-18:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.22","timestamp":"2021-03-11T00:49:18.392936703Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423758.394949_193.1","section":"bucket","name":"nolznk-18","timestamp":"2021-03-11T00:49:18.394949665Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mGlZ5EZiAMG_pDJLlo6k4Ao","ver":1},"status":{"status":"write"}}},{"id":"1_1615423758.399367_194.1","section":"bucket","name":"nolznk-18","timestamp":"2021-03-11T00:49:18.399367098Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mGlZ5EZiAMG_pDJLlo6k4Ao","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423758.430274_195.1","section":"bucket.instance","name":"nolznk-17:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.21","timestamp":"2021-03-11T00:49:18.430274655Z","data":{"read_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":1},"write_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":2},"status":{"status":"write"}}},{"id":"1_1615423758.436085_196.1","section":"bucket.instance","name":"nolznk-17:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.21","timestamp":"2021-03-11T00:49:18.436085903Z","data":{"read_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":1},"write_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423758.449253_197.1","section":"bucket.instance","name":"nolznk-17:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.21","timestamp":"2021-03-11T00:49:18.449253989Z","data":{"read_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":2},"write_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":3},"status":{"status":"write"}}},{"id":"1_1615423758.454852_198.1","section":"bucket.instance","name":"nolznk-17:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.21","timestamp":"2021-03-11T00:49:18.454852873Z","data":{"read_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":2},"write_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":3},"status":{"status":"complete"}}},{"id":"1_1615423758.463142_199.1","section":"bucket.instance","name":"nolznk-18:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.22","timestamp":"2021-03-11T00:49:18.463142338Z","data":{"read_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":1},"write_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":2},"status":{"status":"write"}}},{"id":"1_1615423758.469743_200.1","section":"bucket.instance","name":"nolznk-18:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.22","timestamp":"2021-03-11T00:49:18.469743017Z","data":{"read_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":1},"write_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423758.483410_201.1","section":"bucket.instance","name":"nolznk-18:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.22","timestamp":"2021-03-11T00:49:18.483410006Z","data":{"read_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":2},"write_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":3},"status":{"status":"write"}}},{"id":"1_1615423758.490731_202.1","section":"bucket.instance","name":"nolznk-18:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.22","timestamp":"2021-03-11T00:49:18.490731583Z","data":{"read_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":2},"write_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":3},"status":{"status":"complete"}}},{"id":"1_1615423773.188516_203.1","section":"bucket.instance","name":"nolznk-19:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.23","timestamp":"2021-03-11T00:49:33.188516972Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_bfh_EH3cSHh3fV_umZvAZMT","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.193178_211.1","section":"bucket.instance","name":"nolznk-19:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.23","timestamp":"2021-03-11T00:49:33.193178769Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_bfh_EH3cSHh3fV_umZvAZMT","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.194917_212.1","section":"bucket","name":"nolznk-19","timestamp":"2021-03-11T00:49:33.194917296Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_e0eRqOyRBg-S5yTJnjW1lLD","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.199966_213.1","section":"bucket","name":"nolznk-19","timestamp":"2021-03-11T00:49:33.199966925Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_e0eRqOyRBg-S5yTJnjW1lLD","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.219722_214.1","section":"bucket.instance","name":"nolznk-20:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.24","timestamp":"2021-03-11T00:49:33.219722017Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_tROxQwkgsHmcIV2tAtYwLsg","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.225185_215.1","section":"bucket.instance","name":"nolznk-20:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.24","timestamp":"2021-03-11T00:49:33.225185162Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_tROxQwkgsHmcIV2tAtYwLsg","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.226995_216.1","section":"bucket","name":"nolznk-20","timestamp":"2021-03-11T00:49:33.226995917Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_yIdp1ucJJZI_wIl7WJeHezY","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.233185_217.1","section":"bucket","name":"nolznk-20","timestamp":"2021-03-11T00:49:33.233185260Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_yIdp1ucJJZI_wIl7WJeHezY","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.256444_218.1","section":"bucket.instance","name":"nolznk-19:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.23","timestamp":"2021-03-11T00:49:33.256444610Z","data":{"read_version":{"tag":"_bfh_EH3cSHh3fV_umZvAZMT","ver":1},"write_version":{"tag":"_bfh_EH3cSHh3fV_umZvAZMT","ver":2},"status":{"status":"write"}}},{"id":"1_1615423773.262956_219.1","section":"bucket.instance","name":"nolznk-19:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.23","timestamp":"2021-03-11T00:49:33.262956244Z","data":{"read_version":{"tag":"_bfh_EH3cSHh3fV_umZvAZMT","ver":1},"write_version":{"tag":"_bfh_EH3cSHh3fV_umZvAZMT","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423773.314286_220.1","section":"bucket.instance","name":"nolznk-20:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.24","timestamp":"2021-03-11T00:49:33.314286791Z","data":{"read_version":{"tag":"_tROxQwkgsHmcIV2tAtYwLsg","ver":1},"write_version":{"tag":"_tROxQwkgsHmcIV2tAtYwLsg","ver":2},"status":{"status":"write"}}},{"id":"1_1615423773.322365_221.1","section":"bucket.instance","name":"nolznk-20:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.24","timestamp":"2021-03-11T00:49:33.322365087Z","data":{"read_version":{"tag":"_tROxQwkgsHmcIV2tAtYwLsg","ver":1},"write_version":{"tag":"_tROxQwkgsHmcIV2tAtYwLsg","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423773.388830_222.1","section":"bucket.instance","name":"nolznk-21:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.25","timestamp":"2021-03-11T00:49:33.388830385Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DtCBNb6yWQJ9bljYZUL-Tpd","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.393488_223.1","section":"bucket.instance","name":"nolznk-21:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.25","timestamp":"2021-03-11T00:49:33.393488957Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DtCBNb6yWQJ9bljYZUL-Tpd","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.394906_224.1","section":"bucket","name":"nolznk-21","timestamp":"2021-03-11T00:49:33.394906992Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_V6Lv5bVW3vMXiLEcXPgSdoy","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.398672_225.1","section":"bucket","name":"nolznk-21","timestamp":"2021-03-11T00:49:33.398672730Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_V6Lv5bVW3vMXiLEcXPgSdoy","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.413829_226.1","section":"bucket.instance","name":"nolznk-22:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.26","timestamp":"2021-03-11T00:49:33.413829448Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VqPhf4iuPtHx-nZ4zTRaSwu","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.418512_227.1","section":"bucket.instance","name":"nolznk-22:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.26","timestamp":"2021-03-11T00:49:33.418512596Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VqPhf4iuPtHx-nZ4zTRaSwu","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.420358_228.1","section":"bucket","name":"nolznk-22","timestamp":"2021-03-11T00:49:33.420358132Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_TQxPB4z8uyCZjrmcrjABQiV","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.425060_229.1","section":"bucket","name":"nolznk-22","timestamp":"2021-03-11T00:49:33.425060654Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_TQxPB4z8uyCZjrmcrjABQiV","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.450023_230.1","section":"bucket.instance","name":"nolznk-21:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.25","timestamp":"2021-03-11T00:49:33.450023398Z","data":{"read_version":{"tag":"_DtCBNb6yWQJ9bljYZUL-Tpd","ver":1},"write_version":{"tag":"_DtCBNb6yWQJ9bljYZUL-Tpd","ver":2},"status":{"status":"write"}}},{"id":"1_1615423773.456451_231.1","section":"bucket.instance","name":"nolznk-21:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.25","timestamp":"2021-03-11T00:49:33.456451795Z","data":{"read_version":{"tag":"_DtCBNb6yWQJ9bljYZUL-Tpd","ver":1},"write_version":{"tag":"_DtCBNb6yWQJ9bljYZUL-Tpd","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423773.511182_232.1","section":"bucket.instance","name":"nolznk-22:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.26","timestamp":"2021-03-11T00:49:33.511182120Z","data":{"read_version":{"tag":"_VqPhf4iuPtHx-nZ4zTRaSwu","ver":1},"write_version":{"tag":"_VqPhf4iuPtHx-nZ4zTRaSwu","ver":2},"status":{"status":"write"}}},{"id":"1_1615423773.519683_233.1","section":"bucket.instance","name":"nolznk-22:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.26","timestamp":"2021-03-11T00:49:33.519683540Z","data":{"read_version":{"tag":"_VqPhf4iuPtHx-nZ4zTRaSwu","ver":1},"write_version":{"tag":"_VqPhf4iuPtHx-nZ4zTRaSwu","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423773.595382_234.1","section":"bucket.instance","name":"nolznk-23:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.27","timestamp":"2021-03-11T00:49:33.595382068Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Kd3YMYl8eXvrLGACqaoWSFq","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.600025_235.1","section":"bucket.instance","name":"nolznk-23:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.27","timestamp":"2021-03-11T00:49:33.600025216Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Kd3YMYl8eXvrLGACqaoWSFq","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.601433_236.1","section":"bucket","name":"nolznk-23","timestamp":"2021-03-11T00:49:33.601433770Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_3itXKzsUPhzhes2xUErh0Qx","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.604979_237.1","section":"bucket","name":"nolznk-23","timestamp":"2021-03-11T00:49:33.604979959Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_3itXKzsUPhzhes2xUErh0Qx","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.623668_238.1","section":"bucket.instance","name":"nolznk-24:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.28","timestamp":"2021-03-11T00:49:33.623668196Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DEVpH-WkgJI7HpB3UAQY0hj","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.629418_239.1","section":"bucket.instance","name":"nolznk-24:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.28","timestamp":"2021-03-11T00:49:33.629418819Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DEVpH-WkgJI7HpB3UAQY0hj","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.631233_240.1","section":"bucket","name":"nolznk-24","timestamp":"2021-03-11T00:49:33.631233214Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zWzxEZawxCPuZRL3VFOI87f","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.638186_241.1","section":"bucket","name":"nolznk-24","timestamp":"2021-03-11T00:49:33.638186236Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zWzxEZawxCPuZRL3VFOI87f","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.661473_242.1","section":"bucket.instance","name":"nolznk-23:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.27","timestamp":"2021-03-11T00:49:33.661473290Z","data":{"read_version":{"tag":"_Kd3YMYl8eXvrLGACqaoWSFq","ver":1},"write_version":{"tag":"_Kd3YMYl8eXvrLGACqaoWSFq","ver":2},"status":{"status":"write"}}},{"id":"1_1615423773.666810_243.1","section":"bucket.instance","name":"nolznk-23:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.27","timestamp":"2021-03-11T00:49:33.666810087Z","data":{"read_version":{"tag":"_Kd3YMYl8eXvrLGACqaoWSFq","ver":1},"write_version":{"tag":"_Kd3YMYl8eXvrLGACqaoWSFq","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423774.723261_244.1","section":"bucket.instance","name":"nolznk-24:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.28","timestamp":"2021-03-11T00:49:34.723261077Z","data":{"read_version":{"tag":"_DEVpH-WkgJI7HpB3UAQY0hj","ver":1},"write_version":{"tag":"_DEVpH-WkgJI7HpB3UAQY0hj","ver":2},"status":{"status":"write"}}},{"id":"1_1615423774.732056_245.1","section":"bucket.instance","name":"nolznk-24:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.28","timestamp":"2021-03-11T00:49:34.732056339Z","data":{"read_version":{"tag":"_DEVpH-WkgJI7HpB3UAQY0hj","ver":1},"write_version":{"tag":"_DEVpH-WkgJI7HpB3UAQY0hj","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423774.807833_246.1","section":"bucket.instance","name":"nolznk-25:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.29","timestamp":"2021-03-11T00:49:34.807833043Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_K4b3tWotdt3YxAzVMCBXaXJ","ver":1},"status":{"status":"write"}}},{"id":"1_1615423774.812726_247.1","section":"bucket.instance","name":"nolznk-25:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.29","timestamp":"2021-03-11T00:49:34.812726730Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_K4b3tWotdt3YxAzVMCBXaXJ","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423774.814456_248.1","section":"bucket","name":"nolznk-25","timestamp":"2021-03-11T00:49:34.814456608Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_CGAgOQYApDaSMClQOdo1ewR","ver":1},"status":{"status":"write"}}},{"id":"1_1615423774.819187_249.1","section":"bucket","name":"nolznk-25","timestamp":"2021-03-11T00:49:34.819187613Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_CGAgOQYApDaSMClQOdo1ewR","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423774.837944_250.1","section":"bucket.instance","name":"nolznk-26:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.30","timestamp":"2021-03-11T00:49:34.837944290Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_E3IIKzX9z59ojuvBjbsF3pq","ver":1},"status":{"status":"write"}}},{"id":"1_1615423774.842943_251.1","section":"bucket.instance","name":"nolznk-26:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.30","timestamp":"2021-03-11T00:49:34.842943745Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_E3IIKzX9z59ojuvBjbsF3pq","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423774.844805_252.1","section":"bucket","name":"nolznk-26","timestamp":"2021-03-11T00:49:34.844805420Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_e5EhGvIChq8be4lQdVIoFrQ","ver":1},"status":{"status":"write"}}},{"id":"1_1615423774.849040_253.1","section":"bucket","name":"nolznk-26","timestamp":"2021-03-11T00:49:34.849040958Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_e5EhGvIChq8be4lQdVIoFrQ","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423775.964068_256.1","section":"bucket.instance","name":"nolznk-27:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.31","timestamp":"2021-03-11T00:49:35.964068040Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Rzrci_t2792wGHI9TE4HTRC","ver":1},"status":{"status":"write"}}},{"id":"1_1615423775.969096_257.1","section":"bucket.instance","name":"nolznk-27:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.31","timestamp":"2021-03-11T00:49:35.969096591Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Rzrci_t2792wGHI9TE4HTRC","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423775.970963_258.1","section":"bucket","name":"nolznk-27","timestamp":"2021-03-11T00:49:35.970963524Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_O-sq5_XFIRPaIH9vjf7rIT7","ver":1},"status":{"status":"write"}}},{"id":"1_1615423775.975158_259.1","section":"bucket","name":"nolznk-27","timestamp":"2021-03-11T00:49:35.975158527Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_O-sq5_XFIRPaIH9vjf7rIT7","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423775.995326_260.1","section":"bucket.instance","name":"nolznk-28:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.32","timestamp":"2021-03-11T00:49:35.995326177Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_GdEznOSqpGleDhPhmiqDwDN","ver":1},"status":{"status":"write"}}},{"id":"1_1615423776.000972_261.1","section":"bucket.instance","name":"nolznk-28:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.32","timestamp":"2021-03-11T00:49:36.000972074Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_GdEznOSqpGleDhPhmiqDwDN","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423776.002878_262.1","section":"bucket","name":"nolznk-28","timestamp":"2021-03-11T00:49:36.002878628Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ZHx0nQHYkA_1K9-aju8drea","ver":1},"status":{"status":"write"}}},{"id":"1_1615423776.006942_263.1","section":"bucket","name":"nolznk-28","timestamp":"2021-03-11T00:49:36.006942179Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ZHx0nQHYkA_1K9-aju8drea","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423573.779224_33.1","section":"bucket.instance","name":"nolznk-1:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.1","timestamp":"2021-03-11T00:46:13.779224658Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2t_xvaNhQOSvG3UmDNb-Gmd","ver":1},"status":{"status":"write"}}},{"id":"1_1615423573.785608_34.1","section":"bucket.instance","name":"nolznk-1:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.1","timestamp":"2021-03-11T00:46:13.785608470Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2t_xvaNhQOSvG3UmDNb-Gmd","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423573.787427_35.1","section":"bucket","name":"nolznk-1","timestamp":"2021-03-11T00:46:13.787427525Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NBtj2vMajUaNF8VeKLq1iYO","ver":1},"status":{"status":"write"}}},{"id":"1_1615423573.793388_36.1","section":"bucket","name":"nolznk-1","timestamp":"2021-03-11T00:46:13.793388906Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NBtj2vMajUaNF8VeKLq1iYO","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423573.819219_37.1","section":"bucket.instance","name":"nolznk-2:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.2","timestamp":"2021-03-11T00:46:13.819219101Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gQrL8atL6exiI5rQgMzmgvQ","ver":1},"status":{"status":"write"}}},{"id":"1_1615423573.824837_38.1","section":"bucket.instance","name":"nolznk-2:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.2","timestamp":"2021-03-11T00:46:13.824837055Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gQrL8atL6exiI5rQgMzmgvQ","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423573.826464_39.1","section":"bucket","name":"nolznk-2","timestamp":"2021-03-11T00:46:13.826464055Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_C6tTZv7dm-BiN39XejTafFV","ver":1},"status":{"status":"write"}}},{"id":"1_1615423573.832639_40.1","section":"bucket","name":"nolznk-2","timestamp":"2021-03-11T00:46:13.832639639Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_C6tTZv7dm-BiN39XejTafFV","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423583.648205_41.1","section":"bucket.instance","name":"nolznk-3:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.3","timestamp":"2021-03-11T00:46:23.648205263Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FGN9jBB7YE_ghkHcU1I3LNt","ver":1},"status":{"status":"write"}}},{"id":"1_1615423583.653383_42.1","section":"bucket.instance","name":"nolznk-3:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.3","timestamp":"2021-03-11T00:46:23.653383372Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FGN9jBB7YE_ghkHcU1I3LNt","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423583.655101_43.1","section":"bucket","name":"nolznk-3","timestamp":"2021-03-11T00:46:23.655101249Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_70McMugXBxcL_bL1Cmifz6L","ver":1},"status":{"status":"write"}}},{"id":"1_1615423583.659946_44.1","section":"bucket","name":"nolznk-3","timestamp":"2021-03-11T00:46:23.659946393Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_70McMugXBxcL_bL1Cmifz6L","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423583.679743_45.1","section":"bucket.instance","name":"nolznk-4:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.4","timestamp":"2021-03-11T00:46:23.679743936Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Gr35hKIaC-zpMhxqBvTX4Nu","ver":1},"status":{"status":"write"}}},{"id":"1_1615423583.686290_46.1","section":"bucket.instance","name":"nolznk-4:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.4","timestamp":"2021-03-11T00:46:23.686290426Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Gr35hKIaC-zpMhxqBvTX4Nu","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423583.688328_47.1","section":"bucket","name":"nolznk-4","timestamp":"2021-03-11T00:46:23.688328527Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LCs2kQqpSnxfkqp77rlA4Uh","ver":1},"status":{"status":"write"}}},{"id":"1_1615423583.694980_48.1","section":"bucket","name":"nolznk-4","timestamp":"2021-03-11T00:46:23.694980993Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LCs2kQqpSnxfkqp77rlA4Uh","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423589.811650_51.1","section":"bucket.instance","name":"nolznk-3:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.5","timestamp":"2021-03-11T00:46:29.811650218Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4UUEYyix76dlhi5gQ96jeZO","ver":1},"status":{"status":"write"}}},{"id":"1_1615423589.819550_52.1","section":"bucket.instance","name":"nolznk-3:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.5","timestamp":"2021-03-11T00:46:29.819550644Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4UUEYyix76dlhi5gQ96jeZO","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423589.821577_53.1","section":"bucket","name":"nolznk-3","timestamp":"2021-03-11T00:46:29.821577182Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_nNni5geDzI_cILaAduVvArT","ver":1},"status":{"status":"write"}}},{"id":"1_1615423589.826156_54.1","section":"bucket","name":"nolznk-3","timestamp":"2021-03-11T00:46:29.826156290Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_nNni5geDzI_cILaAduVvArT","ver":1},"status":{"status":"abort"}}},{"id":"1_1615423589.855852_55.1","section":"bucket.instance","name":"nolznk-4:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.6","timestamp":"2021-03-11T00:46:29.855852336Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_uL4uyLLlOHfXnXpokAAu-15","ver":1},"status":{"status":"write"}}},{"id":"1_1615423589.860662_56.1","section":"bucket.instance","name":"nolznk-4:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.6","timestamp":"2021-03-11T00:46:29.860662818Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_uL4uyLLlOHfXnXpokAAu-15","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423589.862946_57.1","section":"bucket","name":"nolznk-4","timestamp":"2021-03-11T00:46:29.862946094Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_C3MbfKZdBWsojefkVvMyhqL","ver":1},"status":{"status":"write"}}},{"id":"1_1615423589.865927_58.1","section":"bucket","name":"nolznk-4","timestamp":"2021-03-11T00:46:29.865927078Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_C3MbfKZdBWsojefkVvMyhqL","ver":1},"status":{"status":"abort"}}},{"id":"1_1615423589.892960_60.1","section":"bucket.instance","name":"nolznk-3:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.7","timestamp":"2021-03-11T00:46:29.892960158Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JW05anS1Np_EfOSPeKyf_yM","ver":1},"status":{"status":"write"}}},{"id":"1_1615423589.899828_61.1","section":"bucket.instance","name":"nolznk-3:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.7","timestamp":"2021-03-11T00:46:29.899828307Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JW05anS1Np_EfOSPeKyf_yM","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423589.901666_62.1","section":"bucket","name":"nolznk-3","timestamp":"2021-03-11T00:46:29.901666754Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_5-d9Q2sYdsRMLXMOsdZg1E-","ver":1},"status":{"status":"write"}}},{"id":"1_1615423589.906825_63.1","section":"bucket","name":"nolznk-3","timestamp":"2021-03-11T00:46:29.906825704Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_5-d9Q2sYdsRMLXMOsdZg1E-","ver":1},"status":{"status":"abort"}}},{"id":"1_1615423589.954490_64.1","section":"bucket.instance","name":"nolznk-4:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.8","timestamp":"2021-03-11T00:46:29.954490622Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_EoLm3bRg_s7LjIXIXwQ2bGl","ver":1},"status":{"status":"write"}}},{"id":"1_1615423589.961897_65.1","section":"bucket.instance","name":"nolznk-4:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.8","timestamp":"2021-03-11T00:46:29.961897362Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_EoLm3bRg_s7LjIXIXwQ2bGl","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423589.963929_66.1","section":"bucket","name":"nolznk-4","timestamp":"2021-03-11T00:46:29.963929759Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_m6SQ98L0z2c9tpG7mwZpVCr","ver":1},"status":{"status":"write"}}},{"id":"1_1615423589.969156_67.1","section":"bucket","name":"nolznk-4","timestamp":"2021-03-11T00:46:29.969156315Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_m6SQ98L0z2c9tpG7mwZpVCr","ver":1},"status":{"status":"abort"}}},{"id":"1_1615423596.160393_71.1","section":"bucket.instance","name":"nolznk-5:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.9","timestamp":"2021-03-11T00:46:36.160393574Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_abeZIu_LDX-6fyUsj8leiUH","ver":1},"status":{"status":"write"}}},{"id":"1_1615423596.166034_72.1","section":"bucket.instance","name":"nolznk-5:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.9","timestamp":"2021-03-11T00:46:36.166034845Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_abeZIu_LDX-6fyUsj8leiUH","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423596.168075_73.1","section":"bucket","name":"nolznk-5","timestamp":"2021-03-11T00:46:36.168075697Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_OPM8B0wzAs7xqCYsHKqP2IP","ver":1},"status":{"status":"write"}}},{"id":"1_1615423596.172637_74.1","section":"bucket","name":"nolznk-5","timestamp":"2021-03-11T00:46:36.172637219Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_OPM8B0wzAs7xqCYsHKqP2IP","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423596.188375_75.1","section":"bucket.instance","name":"nolznk-6:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.10","timestamp":"2021-03-11T00:46:36.188375006Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_CKdgBxyJNwgGTATsCGLE-xq","ver":1},"status":{"status":"write"}}},{"id":"1_1615423596.193886_76.1","section":"bucket.instance","name":"nolznk-6:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.10","timestamp":"2021-03-11T00:46:36.193886239Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_CKdgBxyJNwgGTATsCGLE-xq","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423596.195668_77.1","section":"bucket","name":"nolznk-6","timestamp":"2021-03-11T00:46:36.195668270Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_t15sglXCLoz3bXTGemUXlX_","ver":1},"status":{"status":"write"}}},{"id":"1_1615423596.201627_78.1","section":"bucket","name":"nolznk-6","timestamp":"2021-03-11T00:46:36.201627546Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_t15sglXCLoz3bXTGemUXlX_","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423602.311057_79.1","section":"bucket","name":"nolznk-5","timestamp":"2021-03-11T00:46:42.311057643Z","data":{"read_version":{"tag":"_OPM8B0wzAs7xqCYsHKqP2IP","ver":1},"write_version":{"tag":"_OPM8B0wzAs7xqCYsHKqP2IP","ver":2},"status":{"status":"remove"}}},{"id":"1_1615423602.316644_80.1","section":"bucket","name":"nolznk-5","timestamp":"2021-03-11T00:46:42.316644434Z","data":{"read_version":{"tag":"_OPM8B0wzAs7xqCYsHKqP2IP","ver":1},"write_version":{"tag":"_OPM8B0wzAs7xqCYsHKqP2IP","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423602.356260_81.1","section":"bucket","name":"nolznk-6","timestamp":"2021-03-11T00:46:42.356260413Z","data":{"read_version":{"tag":"_t15sglXCLoz3bXTGemUXlX_","ver":1},"write_version":{"tag":"_t15sglXCLoz3bXTGemUXlX_","ver":2},"status":{"status":"remove"}}},{"id":"1_1615423602.361914_82.1","section":"bucket","name":"nolznk-6","timestamp":"2021-03-11T00:46:42.361914200Z","data":{"read_version":{"tag":"_t15sglXCLoz3bXTGemUXlX_","ver":1},"write_version":{"tag":"_t15sglXCLoz3bXTGemUXlX_","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423608.500814_83.1","section":"bucket.instance","name":"nolznk-7:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.11","timestamp":"2021-03-11T00:46:48.500814832Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_5q5LTZ7iRVPMyiqmZBWgMZZ","ver":1},"status":{"status":"write"}}},{"id":"1_1615423608.506444_84.1","section":"bucket.instance","name":"nolznk-7:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.11","timestamp":"2021-03-11T00:46:48.506444397Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_5q5LTZ7iRVPMyiqmZBWgMZZ","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423608.507991_85.1","section":"bucket","name":"nolznk-7","timestamp":"2021-03-11T00:46:48.507991136Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Fzog3gQiW8a5jzyMWvnY65O","ver":1},"status":{"status":"write"}}},{"id":"1_1615423608.511544_86.1","section":"bucket","name":"nolznk-7","timestamp":"2021-03-11T00:46:48.511544979Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Fzog3gQiW8a5jzyMWvnY65O","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423608.527511_87.1","section":"bucket.instance","name":"nolznk-8:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.12","timestamp":"2021-03-11T00:46:48.527511620Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_yLo2o4q2UdLAGI0c-nBeaK-","ver":1},"status":{"status":"write"}}},{"id":"1_1615423608.532152_88.1","section":"bucket.instance","name":"nolznk-8:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.12","timestamp":"2021-03-11T00:46:48.532152451Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_yLo2o4q2UdLAGI0c-nBeaK-","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423608.533931_89.1","section":"bucket","name":"nolznk-8","timestamp":"2021-03-11T00:46:48.533931862Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_tCC2Km0qZUmiqiViuLUsIA0","ver":1},"status":{"status":"write"}}},{"id":"1_1615423608.538319_90.1","section":"bucket","name":"nolznk-8","timestamp":"2021-03-11T00:46:48.538319128Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_tCC2Km0qZUmiqiViuLUsIA0","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423619.023584_98.1","section":"bucket.instance","name":"nolznk-9:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.13","timestamp":"2021-03-11T00:46:59.023584771Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_tAf18T7A-WGzT8ffKm9GGgu","ver":1},"status":{"status":"write"}}},{"id":"1_1615423619.028138_99.1","section":"bucket.instance","name":"nolznk-9:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.13","timestamp":"2021-03-11T00:46:59.028138621Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_tAf18T7A-WGzT8ffKm9GGgu","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423619.029573_100.1","section":"bucket","name":"nolznk-9","timestamp":"2021-03-11T00:46:59.029573417Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Yu95E0IeKnKtPiXbPggWQ80","ver":1},"status":{"status":"write"}}},{"id":"1_1615423619.033309_101.1","section":"bucket","name":"nolznk-9","timestamp":"2021-03-11T00:46:59.033309932Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Yu95E0IeKnKtPiXbPggWQ80","ver":1},"status":{"status":"complete"}}}] 2021-03-11T00:51:53.441 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '470410ef-8c66-4339-a004-064d3e2c6c91', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:53.442 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '470410ef-8c66-4339-a004-064d3e2c6c91', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:53.442 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 470410ef-8c66-4339-a004-064d3e2c6c91 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:51:53.520 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:51:53.521 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:51:53.711 INFO:teuthology.orchestra.run.smithi124.stdout:[{"id":"1_1615423453.463276_71.1","section":"user","name":"rgw-multisite-test-user","timestamp":"2021-03-11T00:44:13.463276832Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_PZwaTRfFlIhjjRCXuj8pLzU","ver":1},"status":{"status":"write"}}},{"id":"1_1615423453.469908_73.1","section":"user","name":"rgw-multisite-test-user","timestamp":"2021-03-11T00:44:13.469908140Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_PZwaTRfFlIhjjRCXuj8pLzU","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423619.055431_88.1","section":"bucket.instance","name":"nolznk-10:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.14","timestamp":"2021-03-11T00:46:59.055431642Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_x3avBieUj8FdqeKcsgJp8af","ver":1},"status":{"status":"write"}}},{"id":"1_1615423619.060466_92.1","section":"bucket.instance","name":"nolznk-10:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.14","timestamp":"2021-03-11T00:46:59.060466336Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_x3avBieUj8FdqeKcsgJp8af","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423619.062085_93.1","section":"bucket","name":"nolznk-10","timestamp":"2021-03-11T00:46:59.062085564Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_OXNZTe0e4We59jmLs0rMpT_","ver":1},"status":{"status":"write"}}},{"id":"1_1615423619.066252_94.1","section":"bucket","name":"nolznk-10","timestamp":"2021-03-11T00:46:59.066252978Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_OXNZTe0e4We59jmLs0rMpT_","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423640.022370_95.1","section":"bucket.instance","name":"nolznk-11:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.15","timestamp":"2021-03-11T00:47:20.022370354Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_iWEYGp1FEyAGIrnLCQeNoos","ver":1},"status":{"status":"write"}}},{"id":"1_1615423640.027559_104.1","section":"bucket.instance","name":"nolznk-11:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.15","timestamp":"2021-03-11T00:47:20.027559659Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_iWEYGp1FEyAGIrnLCQeNoos","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423640.029576_105.1","section":"bucket","name":"nolznk-11","timestamp":"2021-03-11T00:47:20.029576858Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zpTXMuPNjT5p6d04tpDlMEN","ver":1},"status":{"status":"write"}}},{"id":"1_1615423640.033640_106.1","section":"bucket","name":"nolznk-11","timestamp":"2021-03-11T00:47:20.033640562Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zpTXMuPNjT5p6d04tpDlMEN","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423640.052872_107.1","section":"bucket.instance","name":"nolznk-12:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.16","timestamp":"2021-03-11T00:47:20.052872922Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_p6o2vK-LPluCv4rY02wZXQP","ver":1},"status":{"status":"write"}}},{"id":"1_1615423640.058960_108.1","section":"bucket.instance","name":"nolznk-12:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.16","timestamp":"2021-03-11T00:47:20.058960731Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_p6o2vK-LPluCv4rY02wZXQP","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423640.060616_109.1","section":"bucket","name":"nolznk-12","timestamp":"2021-03-11T00:47:20.060616083Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_px0OaWGnbTRNA9VFAABcem4","ver":1},"status":{"status":"write"}}},{"id":"1_1615423640.064716_110.1","section":"bucket","name":"nolznk-12","timestamp":"2021-03-11T00:47:20.064716045Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_px0OaWGnbTRNA9VFAABcem4","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423640.092752_111.1","section":"bucket.instance","name":"nolznk-11:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.15","timestamp":"2021-03-11T00:47:20.092752724Z","data":{"read_version":{"tag":"_iWEYGp1FEyAGIrnLCQeNoos","ver":1},"write_version":{"tag":"_iWEYGp1FEyAGIrnLCQeNoos","ver":2},"status":{"status":"write"}}},{"id":"1_1615423640.098733_112.1","section":"bucket.instance","name":"nolznk-11:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.15","timestamp":"2021-03-11T00:47:20.098733298Z","data":{"read_version":{"tag":"_iWEYGp1FEyAGIrnLCQeNoos","ver":1},"write_version":{"tag":"_iWEYGp1FEyAGIrnLCQeNoos","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423640.106848_113.1","section":"bucket.instance","name":"nolznk-12:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.16","timestamp":"2021-03-11T00:47:20.106848462Z","data":{"read_version":{"tag":"_p6o2vK-LPluCv4rY02wZXQP","ver":1},"write_version":{"tag":"_p6o2vK-LPluCv4rY02wZXQP","ver":2},"status":{"status":"write"}}},{"id":"1_1615423640.114087_114.1","section":"bucket.instance","name":"nolznk-12:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.16","timestamp":"2021-03-11T00:47:20.114087357Z","data":{"read_version":{"tag":"_p6o2vK-LPluCv4rY02wZXQP","ver":1},"write_version":{"tag":"_p6o2vK-LPluCv4rY02wZXQP","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423679.987558_115.1","section":"bucket.instance","name":"nolznk-13:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.17","timestamp":"2021-03-11T00:47:59.987558340Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gOLKrQmzpEIXiaID7B7wPti","ver":1},"status":{"status":"write"}}},{"id":"1_1615423679.992413_133.1","section":"bucket.instance","name":"nolznk-13:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.17","timestamp":"2021-03-11T00:47:59.992413924Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gOLKrQmzpEIXiaID7B7wPti","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423679.994531_134.1","section":"bucket","name":"nolznk-13","timestamp":"2021-03-11T00:47:59.994531075Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rbJbxoLyu4bIb9mBPNMUjAn","ver":1},"status":{"status":"write"}}},{"id":"1_1615423679.998997_135.1","section":"bucket","name":"nolznk-13","timestamp":"2021-03-11T00:47:59.998997714Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rbJbxoLyu4bIb9mBPNMUjAn","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423680.004552_136.1","section":"bucket.instance","name":"nolznk-13:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.17","timestamp":"2021-03-11T00:48:00.004552915Z","data":{"read_version":{"tag":"_gOLKrQmzpEIXiaID7B7wPti","ver":1},"write_version":{"tag":"_gOLKrQmzpEIXiaID7B7wPti","ver":2},"status":{"status":"write"}}},{"id":"1_1615423680.010411_137.1","section":"bucket.instance","name":"nolznk-13:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.17","timestamp":"2021-03-11T00:48:00.010411929Z","data":{"read_version":{"tag":"_gOLKrQmzpEIXiaID7B7wPti","ver":1},"write_version":{"tag":"_gOLKrQmzpEIXiaID7B7wPti","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423703.095803_138.1","section":"bucket.instance","name":"nolznk-14:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.18","timestamp":"2021-03-11T00:48:23.095803789Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":1},"status":{"status":"write"}}},{"id":"1_1615423703.100241_149.1","section":"bucket.instance","name":"nolznk-14:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.18","timestamp":"2021-03-11T00:48:23.100241785Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423703.102699_150.1","section":"bucket","name":"nolznk-14","timestamp":"2021-03-11T00:48:23.102699204Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rwmuSTK5NpKSOt_xCqzBBH-","ver":1},"status":{"status":"write"}}},{"id":"1_1615423703.106524_151.1","section":"bucket","name":"nolznk-14","timestamp":"2021-03-11T00:48:23.106524997Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rwmuSTK5NpKSOt_xCqzBBH-","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423716.262443_152.1","section":"bucket.instance","name":"nolznk-14:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.18","timestamp":"2021-03-11T00:48:36.262443475Z","data":{"read_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":1},"write_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":2},"status":{"status":"write"}}},{"id":"1_1615423716.269301_157.1","section":"bucket.instance","name":"nolznk-14:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.18","timestamp":"2021-03-11T00:48:36.269301193Z","data":{"read_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":1},"write_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423729.605376_158.1","section":"bucket.instance","name":"nolznk-14:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.18","timestamp":"2021-03-11T00:48:49.605376112Z","data":{"read_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":2},"write_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":3},"status":{"status":"write"}}},{"id":"1_1615423729.613265_163.1","section":"bucket.instance","name":"nolznk-14:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.18","timestamp":"2021-03-11T00:48:49.613265598Z","data":{"read_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":2},"write_version":{"tag":"_3_euJS8rjPrbmnafolqQ0WS","ver":3},"status":{"status":"complete"}}},{"id":"1_1615423743.264880_164.1","section":"bucket.instance","name":"nolznk-15:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.19","timestamp":"2021-03-11T00:49:03.264880629Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ObhA4ynS0ZkzOShAmXyPpuK","ver":1},"status":{"status":"write"}}},{"id":"1_1615423743.270054_169.1","section":"bucket.instance","name":"nolznk-15:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.19","timestamp":"2021-03-11T00:49:03.270054481Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ObhA4ynS0ZkzOShAmXyPpuK","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423743.271574_170.1","section":"bucket","name":"nolznk-15","timestamp":"2021-03-11T00:49:03.271574605Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_GsoaEGYqmDbKVeNP79Se_1m","ver":1},"status":{"status":"write"}}},{"id":"1_1615423743.275888_171.1","section":"bucket","name":"nolznk-15","timestamp":"2021-03-11T00:49:03.275888088Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_GsoaEGYqmDbKVeNP79Se_1m","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423743.298858_172.1","section":"bucket.instance","name":"nolznk-16:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.20","timestamp":"2021-03-11T00:49:03.298858602Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_AOlGazAgHoexzcjIgFKH-Mx","ver":1},"status":{"status":"write"}}},{"id":"1_1615423743.303694_173.1","section":"bucket.instance","name":"nolznk-16:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.20","timestamp":"2021-03-11T00:49:03.303694132Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_AOlGazAgHoexzcjIgFKH-Mx","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423743.305295_174.1","section":"bucket","name":"nolznk-16","timestamp":"2021-03-11T00:49:03.305295956Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Ems82ocmFs3LsMnJXhG2suw","ver":1},"status":{"status":"write"}}},{"id":"1_1615423743.309279_175.1","section":"bucket","name":"nolznk-16","timestamp":"2021-03-11T00:49:03.309279431Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Ems82ocmFs3LsMnJXhG2suw","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423743.335935_176.1","section":"bucket.instance","name":"nolznk-15:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.19","timestamp":"2021-03-11T00:49:03.335935935Z","data":{"read_version":{"tag":"_ObhA4ynS0ZkzOShAmXyPpuK","ver":1},"write_version":{"tag":"_ObhA4ynS0ZkzOShAmXyPpuK","ver":2},"status":{"status":"write"}}},{"id":"1_1615423743.341225_177.1","section":"bucket.instance","name":"nolznk-15:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.19","timestamp":"2021-03-11T00:49:03.341225643Z","data":{"read_version":{"tag":"_ObhA4ynS0ZkzOShAmXyPpuK","ver":1},"write_version":{"tag":"_ObhA4ynS0ZkzOShAmXyPpuK","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423743.351429_178.1","section":"bucket.instance","name":"nolznk-16:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.20","timestamp":"2021-03-11T00:49:03.351429534Z","data":{"read_version":{"tag":"_AOlGazAgHoexzcjIgFKH-Mx","ver":1},"write_version":{"tag":"_AOlGazAgHoexzcjIgFKH-Mx","ver":2},"status":{"status":"write"}}},{"id":"1_1615423743.357959_179.1","section":"bucket.instance","name":"nolznk-16:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.20","timestamp":"2021-03-11T00:49:03.357959007Z","data":{"read_version":{"tag":"_AOlGazAgHoexzcjIgFKH-Mx","ver":1},"write_version":{"tag":"_AOlGazAgHoexzcjIgFKH-Mx","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423758.356100_180.1","section":"bucket.instance","name":"nolznk-17:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.21","timestamp":"2021-03-11T00:49:18.356100134Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":1},"status":{"status":"write"}}},{"id":"1_1615423758.360834_188.1","section":"bucket.instance","name":"nolznk-17:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.21","timestamp":"2021-03-11T00:49:18.360834148Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423758.362590_189.1","section":"bucket","name":"nolznk-17","timestamp":"2021-03-11T00:49:18.362590752Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_poFzmuQa6qComI9TTZ1EQU6","ver":1},"status":{"status":"write"}}},{"id":"1_1615423758.366352_190.1","section":"bucket","name":"nolznk-17","timestamp":"2021-03-11T00:49:18.366352196Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_poFzmuQa6qComI9TTZ1EQU6","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423758.387291_191.1","section":"bucket.instance","name":"nolznk-18:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.22","timestamp":"2021-03-11T00:49:18.387291130Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":1},"status":{"status":"write"}}},{"id":"1_1615423758.392936_192.1","section":"bucket.instance","name":"nolznk-18:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.22","timestamp":"2021-03-11T00:49:18.392936703Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423758.394949_193.1","section":"bucket","name":"nolznk-18","timestamp":"2021-03-11T00:49:18.394949665Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mGlZ5EZiAMG_pDJLlo6k4Ao","ver":1},"status":{"status":"write"}}},{"id":"1_1615423758.399367_194.1","section":"bucket","name":"nolznk-18","timestamp":"2021-03-11T00:49:18.399367098Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mGlZ5EZiAMG_pDJLlo6k4Ao","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423758.430274_195.1","section":"bucket.instance","name":"nolznk-17:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.21","timestamp":"2021-03-11T00:49:18.430274655Z","data":{"read_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":1},"write_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":2},"status":{"status":"write"}}},{"id":"1_1615423758.436085_196.1","section":"bucket.instance","name":"nolznk-17:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.21","timestamp":"2021-03-11T00:49:18.436085903Z","data":{"read_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":1},"write_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423758.449253_197.1","section":"bucket.instance","name":"nolznk-17:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.21","timestamp":"2021-03-11T00:49:18.449253989Z","data":{"read_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":2},"write_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":3},"status":{"status":"write"}}},{"id":"1_1615423758.454852_198.1","section":"bucket.instance","name":"nolznk-17:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.21","timestamp":"2021-03-11T00:49:18.454852873Z","data":{"read_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":2},"write_version":{"tag":"_DgJw5Co2-bmt8y28aP3D4YI","ver":3},"status":{"status":"complete"}}},{"id":"1_1615423758.463142_199.1","section":"bucket.instance","name":"nolznk-18:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.22","timestamp":"2021-03-11T00:49:18.463142338Z","data":{"read_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":1},"write_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":2},"status":{"status":"write"}}},{"id":"1_1615423758.469743_200.1","section":"bucket.instance","name":"nolznk-18:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.22","timestamp":"2021-03-11T00:49:18.469743017Z","data":{"read_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":1},"write_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423758.483410_201.1","section":"bucket.instance","name":"nolznk-18:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.22","timestamp":"2021-03-11T00:49:18.483410006Z","data":{"read_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":2},"write_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":3},"status":{"status":"write"}}},{"id":"1_1615423758.490731_202.1","section":"bucket.instance","name":"nolznk-18:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.22","timestamp":"2021-03-11T00:49:18.490731583Z","data":{"read_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":2},"write_version":{"tag":"_Ta5IASB5EdOERfjLRcTyOI7","ver":3},"status":{"status":"complete"}}},{"id":"1_1615423773.188516_203.1","section":"bucket.instance","name":"nolznk-19:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.23","timestamp":"2021-03-11T00:49:33.188516972Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_bfh_EH3cSHh3fV_umZvAZMT","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.193178_211.1","section":"bucket.instance","name":"nolznk-19:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.23","timestamp":"2021-03-11T00:49:33.193178769Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_bfh_EH3cSHh3fV_umZvAZMT","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.194917_212.1","section":"bucket","name":"nolznk-19","timestamp":"2021-03-11T00:49:33.194917296Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_e0eRqOyRBg-S5yTJnjW1lLD","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.199966_213.1","section":"bucket","name":"nolznk-19","timestamp":"2021-03-11T00:49:33.199966925Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_e0eRqOyRBg-S5yTJnjW1lLD","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.219722_214.1","section":"bucket.instance","name":"nolznk-20:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.24","timestamp":"2021-03-11T00:49:33.219722017Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_tROxQwkgsHmcIV2tAtYwLsg","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.225185_215.1","section":"bucket.instance","name":"nolznk-20:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.24","timestamp":"2021-03-11T00:49:33.225185162Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_tROxQwkgsHmcIV2tAtYwLsg","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.226995_216.1","section":"bucket","name":"nolznk-20","timestamp":"2021-03-11T00:49:33.226995917Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_yIdp1ucJJZI_wIl7WJeHezY","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.233185_217.1","section":"bucket","name":"nolznk-20","timestamp":"2021-03-11T00:49:33.233185260Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_yIdp1ucJJZI_wIl7WJeHezY","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.256444_218.1","section":"bucket.instance","name":"nolznk-19:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.23","timestamp":"2021-03-11T00:49:33.256444610Z","data":{"read_version":{"tag":"_bfh_EH3cSHh3fV_umZvAZMT","ver":1},"write_version":{"tag":"_bfh_EH3cSHh3fV_umZvAZMT","ver":2},"status":{"status":"write"}}},{"id":"1_1615423773.262956_219.1","section":"bucket.instance","name":"nolznk-19:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.23","timestamp":"2021-03-11T00:49:33.262956244Z","data":{"read_version":{"tag":"_bfh_EH3cSHh3fV_umZvAZMT","ver":1},"write_version":{"tag":"_bfh_EH3cSHh3fV_umZvAZMT","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423773.314286_220.1","section":"bucket.instance","name":"nolznk-20:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.24","timestamp":"2021-03-11T00:49:33.314286791Z","data":{"read_version":{"tag":"_tROxQwkgsHmcIV2tAtYwLsg","ver":1},"write_version":{"tag":"_tROxQwkgsHmcIV2tAtYwLsg","ver":2},"status":{"status":"write"}}},{"id":"1_1615423773.322365_221.1","section":"bucket.instance","name":"nolznk-20:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.24","timestamp":"2021-03-11T00:49:33.322365087Z","data":{"read_version":{"tag":"_tROxQwkgsHmcIV2tAtYwLsg","ver":1},"write_version":{"tag":"_tROxQwkgsHmcIV2tAtYwLsg","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423773.388830_222.1","section":"bucket.instance","name":"nolznk-21:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.25","timestamp":"2021-03-11T00:49:33.388830385Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DtCBNb6yWQJ9bljYZUL-Tpd","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.393488_223.1","section":"bucket.instance","name":"nolznk-21:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.25","timestamp":"2021-03-11T00:49:33.393488957Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DtCBNb6yWQJ9bljYZUL-Tpd","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.394906_224.1","section":"bucket","name":"nolznk-21","timestamp":"2021-03-11T00:49:33.394906992Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_V6Lv5bVW3vMXiLEcXPgSdoy","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.398672_225.1","section":"bucket","name":"nolznk-21","timestamp":"2021-03-11T00:49:33.398672730Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_V6Lv5bVW3vMXiLEcXPgSdoy","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.413829_226.1","section":"bucket.instance","name":"nolznk-22:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.26","timestamp":"2021-03-11T00:49:33.413829448Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VqPhf4iuPtHx-nZ4zTRaSwu","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.418512_227.1","section":"bucket.instance","name":"nolznk-22:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.26","timestamp":"2021-03-11T00:49:33.418512596Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VqPhf4iuPtHx-nZ4zTRaSwu","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.420358_228.1","section":"bucket","name":"nolznk-22","timestamp":"2021-03-11T00:49:33.420358132Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_TQxPB4z8uyCZjrmcrjABQiV","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.425060_229.1","section":"bucket","name":"nolznk-22","timestamp":"2021-03-11T00:49:33.425060654Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_TQxPB4z8uyCZjrmcrjABQiV","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.450023_230.1","section":"bucket.instance","name":"nolznk-21:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.25","timestamp":"2021-03-11T00:49:33.450023398Z","data":{"read_version":{"tag":"_DtCBNb6yWQJ9bljYZUL-Tpd","ver":1},"write_version":{"tag":"_DtCBNb6yWQJ9bljYZUL-Tpd","ver":2},"status":{"status":"write"}}},{"id":"1_1615423773.456451_231.1","section":"bucket.instance","name":"nolznk-21:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.25","timestamp":"2021-03-11T00:49:33.456451795Z","data":{"read_version":{"tag":"_DtCBNb6yWQJ9bljYZUL-Tpd","ver":1},"write_version":{"tag":"_DtCBNb6yWQJ9bljYZUL-Tpd","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423773.511182_232.1","section":"bucket.instance","name":"nolznk-22:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.26","timestamp":"2021-03-11T00:49:33.511182120Z","data":{"read_version":{"tag":"_VqPhf4iuPtHx-nZ4zTRaSwu","ver":1},"write_version":{"tag":"_VqPhf4iuPtHx-nZ4zTRaSwu","ver":2},"status":{"status":"write"}}},{"id":"1_1615423773.519683_233.1","section":"bucket.instance","name":"nolznk-22:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.26","timestamp":"2021-03-11T00:49:33.519683540Z","data":{"read_version":{"tag":"_VqPhf4iuPtHx-nZ4zTRaSwu","ver":1},"write_version":{"tag":"_VqPhf4iuPtHx-nZ4zTRaSwu","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423773.595382_234.1","section":"bucket.instance","name":"nolznk-23:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.27","timestamp":"2021-03-11T00:49:33.595382068Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Kd3YMYl8eXvrLGACqaoWSFq","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.600025_235.1","section":"bucket.instance","name":"nolznk-23:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.27","timestamp":"2021-03-11T00:49:33.600025216Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Kd3YMYl8eXvrLGACqaoWSFq","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.601433_236.1","section":"bucket","name":"nolznk-23","timestamp":"2021-03-11T00:49:33.601433770Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_3itXKzsUPhzhes2xUErh0Qx","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.604979_237.1","section":"bucket","name":"nolznk-23","timestamp":"2021-03-11T00:49:33.604979959Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_3itXKzsUPhzhes2xUErh0Qx","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.623668_238.1","section":"bucket.instance","name":"nolznk-24:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.28","timestamp":"2021-03-11T00:49:33.623668196Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DEVpH-WkgJI7HpB3UAQY0hj","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.629418_239.1","section":"bucket.instance","name":"nolznk-24:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.28","timestamp":"2021-03-11T00:49:33.629418819Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DEVpH-WkgJI7HpB3UAQY0hj","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.631233_240.1","section":"bucket","name":"nolznk-24","timestamp":"2021-03-11T00:49:33.631233214Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zWzxEZawxCPuZRL3VFOI87f","ver":1},"status":{"status":"write"}}},{"id":"1_1615423773.638186_241.1","section":"bucket","name":"nolznk-24","timestamp":"2021-03-11T00:49:33.638186236Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zWzxEZawxCPuZRL3VFOI87f","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423773.661473_242.1","section":"bucket.instance","name":"nolznk-23:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.27","timestamp":"2021-03-11T00:49:33.661473290Z","data":{"read_version":{"tag":"_Kd3YMYl8eXvrLGACqaoWSFq","ver":1},"write_version":{"tag":"_Kd3YMYl8eXvrLGACqaoWSFq","ver":2},"status":{"status":"write"}}},{"id":"1_1615423773.666810_243.1","section":"bucket.instance","name":"nolznk-23:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.27","timestamp":"2021-03-11T00:49:33.666810087Z","data":{"read_version":{"tag":"_Kd3YMYl8eXvrLGACqaoWSFq","ver":1},"write_version":{"tag":"_Kd3YMYl8eXvrLGACqaoWSFq","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423774.723261_244.1","section":"bucket.instance","name":"nolznk-24:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.28","timestamp":"2021-03-11T00:49:34.723261077Z","data":{"read_version":{"tag":"_DEVpH-WkgJI7HpB3UAQY0hj","ver":1},"write_version":{"tag":"_DEVpH-WkgJI7HpB3UAQY0hj","ver":2},"status":{"status":"write"}}},{"id":"1_1615423774.732056_245.1","section":"bucket.instance","name":"nolznk-24:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.28","timestamp":"2021-03-11T00:49:34.732056339Z","data":{"read_version":{"tag":"_DEVpH-WkgJI7HpB3UAQY0hj","ver":1},"write_version":{"tag":"_DEVpH-WkgJI7HpB3UAQY0hj","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423774.807833_246.1","section":"bucket.instance","name":"nolznk-25:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.29","timestamp":"2021-03-11T00:49:34.807833043Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_K4b3tWotdt3YxAzVMCBXaXJ","ver":1},"status":{"status":"write"}}},{"id":"1_1615423774.812726_247.1","section":"bucket.instance","name":"nolznk-25:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.29","timestamp":"2021-03-11T00:49:34.812726730Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_K4b3tWotdt3YxAzVMCBXaXJ","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423774.814456_248.1","section":"bucket","name":"nolznk-25","timestamp":"2021-03-11T00:49:34.814456608Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_CGAgOQYApDaSMClQOdo1ewR","ver":1},"status":{"status":"write"}}},{"id":"1_1615423774.819187_249.1","section":"bucket","name":"nolznk-25","timestamp":"2021-03-11T00:49:34.819187613Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_CGAgOQYApDaSMClQOdo1ewR","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423774.837944_250.1","section":"bucket.instance","name":"nolznk-26:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.30","timestamp":"2021-03-11T00:49:34.837944290Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_E3IIKzX9z59ojuvBjbsF3pq","ver":1},"status":{"status":"write"}}},{"id":"1_1615423774.842943_251.1","section":"bucket.instance","name":"nolznk-26:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.30","timestamp":"2021-03-11T00:49:34.842943745Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_E3IIKzX9z59ojuvBjbsF3pq","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423774.844805_252.1","section":"bucket","name":"nolznk-26","timestamp":"2021-03-11T00:49:34.844805420Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_e5EhGvIChq8be4lQdVIoFrQ","ver":1},"status":{"status":"write"}}},{"id":"1_1615423774.849040_253.1","section":"bucket","name":"nolznk-26","timestamp":"2021-03-11T00:49:34.849040958Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_e5EhGvIChq8be4lQdVIoFrQ","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423775.964068_256.1","section":"bucket.instance","name":"nolznk-27:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.31","timestamp":"2021-03-11T00:49:35.964068040Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Rzrci_t2792wGHI9TE4HTRC","ver":1},"status":{"status":"write"}}},{"id":"1_1615423775.969096_257.1","section":"bucket.instance","name":"nolznk-27:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.31","timestamp":"2021-03-11T00:49:35.969096591Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Rzrci_t2792wGHI9TE4HTRC","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423775.970963_258.1","section":"bucket","name":"nolznk-27","timestamp":"2021-03-11T00:49:35.970963524Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_O-sq5_XFIRPaIH9vjf7rIT7","ver":1},"status":{"status":"write"}}},{"id":"1_1615423775.975158_259.1","section":"bucket","name":"nolznk-27","timestamp":"2021-03-11T00:49:35.975158527Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_O-sq5_XFIRPaIH9vjf7rIT7","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423775.995326_260.1","section":"bucket.instance","name":"nolznk-28:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.32","timestamp":"2021-03-11T00:49:35.995326177Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_GdEznOSqpGleDhPhmiqDwDN","ver":1},"status":{"status":"write"}}},{"id":"1_1615423776.000972_261.1","section":"bucket.instance","name":"nolznk-28:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.32","timestamp":"2021-03-11T00:49:36.000972074Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_GdEznOSqpGleDhPhmiqDwDN","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423776.002878_262.1","section":"bucket","name":"nolznk-28","timestamp":"2021-03-11T00:49:36.002878628Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ZHx0nQHYkA_1K9-aju8drea","ver":1},"status":{"status":"write"}}},{"id":"1_1615423776.006942_263.1","section":"bucket","name":"nolznk-28","timestamp":"2021-03-11T00:49:36.006942179Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ZHx0nQHYkA_1K9-aju8drea","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423573.779224_33.1","section":"bucket.instance","name":"nolznk-1:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.1","timestamp":"2021-03-11T00:46:13.779224658Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2t_xvaNhQOSvG3UmDNb-Gmd","ver":1},"status":{"status":"write"}}},{"id":"1_1615423573.785608_34.1","section":"bucket.instance","name":"nolznk-1:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.1","timestamp":"2021-03-11T00:46:13.785608470Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2t_xvaNhQOSvG3UmDNb-Gmd","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423573.787427_35.1","section":"bucket","name":"nolznk-1","timestamp":"2021-03-11T00:46:13.787427525Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NBtj2vMajUaNF8VeKLq1iYO","ver":1},"status":{"status":"write"}}},{"id":"1_1615423573.793388_36.1","section":"bucket","name":"nolznk-1","timestamp":"2021-03-11T00:46:13.793388906Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NBtj2vMajUaNF8VeKLq1iYO","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423573.819219_37.1","section":"bucket.instance","name":"nolznk-2:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.2","timestamp":"2021-03-11T00:46:13.819219101Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gQrL8atL6exiI5rQgMzmgvQ","ver":1},"status":{"status":"write"}}},{"id":"1_1615423573.824837_38.1","section":"bucket.instance","name":"nolznk-2:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.2","timestamp":"2021-03-11T00:46:13.824837055Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gQrL8atL6exiI5rQgMzmgvQ","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423573.826464_39.1","section":"bucket","name":"nolznk-2","timestamp":"2021-03-11T00:46:13.826464055Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_C6tTZv7dm-BiN39XejTafFV","ver":1},"status":{"status":"write"}}},{"id":"1_1615423573.832639_40.1","section":"bucket","name":"nolznk-2","timestamp":"2021-03-11T00:46:13.832639639Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_C6tTZv7dm-BiN39XejTafFV","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423583.648205_41.1","section":"bucket.instance","name":"nolznk-3:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.3","timestamp":"2021-03-11T00:46:23.648205263Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FGN9jBB7YE_ghkHcU1I3LNt","ver":1},"status":{"status":"write"}}},{"id":"1_1615423583.653383_42.1","section":"bucket.instance","name":"nolznk-3:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.3","timestamp":"2021-03-11T00:46:23.653383372Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FGN9jBB7YE_ghkHcU1I3LNt","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423583.655101_43.1","section":"bucket","name":"nolznk-3","timestamp":"2021-03-11T00:46:23.655101249Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_70McMugXBxcL_bL1Cmifz6L","ver":1},"status":{"status":"write"}}},{"id":"1_1615423583.659946_44.1","section":"bucket","name":"nolznk-3","timestamp":"2021-03-11T00:46:23.659946393Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_70McMugXBxcL_bL1Cmifz6L","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423583.679743_45.1","section":"bucket.instance","name":"nolznk-4:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.4","timestamp":"2021-03-11T00:46:23.679743936Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Gr35hKIaC-zpMhxqBvTX4Nu","ver":1},"status":{"status":"write"}}},{"id":"1_1615423583.686290_46.1","section":"bucket.instance","name":"nolznk-4:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.4","timestamp":"2021-03-11T00:46:23.686290426Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Gr35hKIaC-zpMhxqBvTX4Nu","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423583.688328_47.1","section":"bucket","name":"nolznk-4","timestamp":"2021-03-11T00:46:23.688328527Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LCs2kQqpSnxfkqp77rlA4Uh","ver":1},"status":{"status":"write"}}},{"id":"1_1615423583.694980_48.1","section":"bucket","name":"nolznk-4","timestamp":"2021-03-11T00:46:23.694980993Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LCs2kQqpSnxfkqp77rlA4Uh","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423589.811650_51.1","section":"bucket.instance","name":"nolznk-3:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.5","timestamp":"2021-03-11T00:46:29.811650218Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4UUEYyix76dlhi5gQ96jeZO","ver":1},"status":{"status":"write"}}},{"id":"1_1615423589.819550_52.1","section":"bucket.instance","name":"nolznk-3:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.5","timestamp":"2021-03-11T00:46:29.819550644Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4UUEYyix76dlhi5gQ96jeZO","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423589.821577_53.1","section":"bucket","name":"nolznk-3","timestamp":"2021-03-11T00:46:29.821577182Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_nNni5geDzI_cILaAduVvArT","ver":1},"status":{"status":"write"}}},{"id":"1_1615423589.826156_54.1","section":"bucket","name":"nolznk-3","timestamp":"2021-03-11T00:46:29.826156290Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_nNni5geDzI_cILaAduVvArT","ver":1},"status":{"status":"abort"}}},{"id":"1_1615423589.855852_55.1","section":"bucket.instance","name":"nolznk-4:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.6","timestamp":"2021-03-11T00:46:29.855852336Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_uL4uyLLlOHfXnXpokAAu-15","ver":1},"status":{"status":"write"}}},{"id":"1_1615423589.860662_56.1","section":"bucket.instance","name":"nolznk-4:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.6","timestamp":"2021-03-11T00:46:29.860662818Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_uL4uyLLlOHfXnXpokAAu-15","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423589.862946_57.1","section":"bucket","name":"nolznk-4","timestamp":"2021-03-11T00:46:29.862946094Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_C3MbfKZdBWsojefkVvMyhqL","ver":1},"status":{"status":"write"}}},{"id":"1_1615423589.865927_58.1","section":"bucket","name":"nolznk-4","timestamp":"2021-03-11T00:46:29.865927078Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_C3MbfKZdBWsojefkVvMyhqL","ver":1},"status":{"status":"abort"}}},{"id":"1_1615423589.892960_60.1","section":"bucket.instance","name":"nolznk-3:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.7","timestamp":"2021-03-11T00:46:29.892960158Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JW05anS1Np_EfOSPeKyf_yM","ver":1},"status":{"status":"write"}}},{"id":"1_1615423589.899828_61.1","section":"bucket.instance","name":"nolznk-3:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.7","timestamp":"2021-03-11T00:46:29.899828307Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JW05anS1Np_EfOSPeKyf_yM","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423589.901666_62.1","section":"bucket","name":"nolznk-3","timestamp":"2021-03-11T00:46:29.901666754Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_5-d9Q2sYdsRMLXMOsdZg1E-","ver":1},"status":{"status":"write"}}},{"id":"1_1615423589.906825_63.1","section":"bucket","name":"nolznk-3","timestamp":"2021-03-11T00:46:29.906825704Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_5-d9Q2sYdsRMLXMOsdZg1E-","ver":1},"status":{"status":"abort"}}},{"id":"1_1615423589.954490_64.1","section":"bucket.instance","name":"nolznk-4:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.8","timestamp":"2021-03-11T00:46:29.954490622Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_EoLm3bRg_s7LjIXIXwQ2bGl","ver":1},"status":{"status":"write"}}},{"id":"1_1615423589.961897_65.1","section":"bucket.instance","name":"nolznk-4:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.8","timestamp":"2021-03-11T00:46:29.961897362Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_EoLm3bRg_s7LjIXIXwQ2bGl","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423589.963929_66.1","section":"bucket","name":"nolznk-4","timestamp":"2021-03-11T00:46:29.963929759Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_m6SQ98L0z2c9tpG7mwZpVCr","ver":1},"status":{"status":"write"}}},{"id":"1_1615423589.969156_67.1","section":"bucket","name":"nolznk-4","timestamp":"2021-03-11T00:46:29.969156315Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_m6SQ98L0z2c9tpG7mwZpVCr","ver":1},"status":{"status":"abort"}}},{"id":"1_1615423596.160393_71.1","section":"bucket.instance","name":"nolznk-5:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.9","timestamp":"2021-03-11T00:46:36.160393574Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_abeZIu_LDX-6fyUsj8leiUH","ver":1},"status":{"status":"write"}}},{"id":"1_1615423596.166034_72.1","section":"bucket.instance","name":"nolznk-5:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.9","timestamp":"2021-03-11T00:46:36.166034845Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_abeZIu_LDX-6fyUsj8leiUH","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423596.168075_73.1","section":"bucket","name":"nolznk-5","timestamp":"2021-03-11T00:46:36.168075697Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_OPM8B0wzAs7xqCYsHKqP2IP","ver":1},"status":{"status":"write"}}},{"id":"1_1615423596.172637_74.1","section":"bucket","name":"nolznk-5","timestamp":"2021-03-11T00:46:36.172637219Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_OPM8B0wzAs7xqCYsHKqP2IP","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423596.188375_75.1","section":"bucket.instance","name":"nolznk-6:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.10","timestamp":"2021-03-11T00:46:36.188375006Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_CKdgBxyJNwgGTATsCGLE-xq","ver":1},"status":{"status":"write"}}},{"id":"1_1615423596.193886_76.1","section":"bucket.instance","name":"nolznk-6:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.10","timestamp":"2021-03-11T00:46:36.193886239Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_CKdgBxyJNwgGTATsCGLE-xq","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423596.195668_77.1","section":"bucket","name":"nolznk-6","timestamp":"2021-03-11T00:46:36.195668270Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_t15sglXCLoz3bXTGemUXlX_","ver":1},"status":{"status":"write"}}},{"id":"1_1615423596.201627_78.1","section":"bucket","name":"nolznk-6","timestamp":"2021-03-11T00:46:36.201627546Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_t15sglXCLoz3bXTGemUXlX_","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423602.311057_79.1","section":"bucket","name":"nolznk-5","timestamp":"2021-03-11T00:46:42.311057643Z","data":{"read_version":{"tag":"_OPM8B0wzAs7xqCYsHKqP2IP","ver":1},"write_version":{"tag":"_OPM8B0wzAs7xqCYsHKqP2IP","ver":2},"status":{"status":"remove"}}},{"id":"1_1615423602.316644_80.1","section":"bucket","name":"nolznk-5","timestamp":"2021-03-11T00:46:42.316644434Z","data":{"read_version":{"tag":"_OPM8B0wzAs7xqCYsHKqP2IP","ver":1},"write_version":{"tag":"_OPM8B0wzAs7xqCYsHKqP2IP","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423602.356260_81.1","section":"bucket","name":"nolznk-6","timestamp":"2021-03-11T00:46:42.356260413Z","data":{"read_version":{"tag":"_t15sglXCLoz3bXTGemUXlX_","ver":1},"write_version":{"tag":"_t15sglXCLoz3bXTGemUXlX_","ver":2},"status":{"status":"remove"}}},{"id":"1_1615423602.361914_82.1","section":"bucket","name":"nolznk-6","timestamp":"2021-03-11T00:46:42.361914200Z","data":{"read_version":{"tag":"_t15sglXCLoz3bXTGemUXlX_","ver":1},"write_version":{"tag":"_t15sglXCLoz3bXTGemUXlX_","ver":2},"status":{"status":"complete"}}},{"id":"1_1615423608.500814_83.1","section":"bucket.instance","name":"nolznk-7:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.11","timestamp":"2021-03-11T00:46:48.500814832Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_5q5LTZ7iRVPMyiqmZBWgMZZ","ver":1},"status":{"status":"write"}}},{"id":"1_1615423608.506444_84.1","section":"bucket.instance","name":"nolznk-7:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.11","timestamp":"2021-03-11T00:46:48.506444397Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_5q5LTZ7iRVPMyiqmZBWgMZZ","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423608.507991_85.1","section":"bucket","name":"nolznk-7","timestamp":"2021-03-11T00:46:48.507991136Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Fzog3gQiW8a5jzyMWvnY65O","ver":1},"status":{"status":"write"}}},{"id":"1_1615423608.511544_86.1","section":"bucket","name":"nolznk-7","timestamp":"2021-03-11T00:46:48.511544979Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Fzog3gQiW8a5jzyMWvnY65O","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423608.527511_87.1","section":"bucket.instance","name":"nolznk-8:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.12","timestamp":"2021-03-11T00:46:48.527511620Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_yLo2o4q2UdLAGI0c-nBeaK-","ver":1},"status":{"status":"write"}}},{"id":"1_1615423608.532152_88.1","section":"bucket.instance","name":"nolznk-8:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.12","timestamp":"2021-03-11T00:46:48.532152451Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_yLo2o4q2UdLAGI0c-nBeaK-","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423608.533931_89.1","section":"bucket","name":"nolznk-8","timestamp":"2021-03-11T00:46:48.533931862Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_tCC2Km0qZUmiqiViuLUsIA0","ver":1},"status":{"status":"write"}}},{"id":"1_1615423608.538319_90.1","section":"bucket","name":"nolznk-8","timestamp":"2021-03-11T00:46:48.538319128Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_tCC2Km0qZUmiqiViuLUsIA0","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423619.023584_98.1","section":"bucket.instance","name":"nolznk-9:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.13","timestamp":"2021-03-11T00:46:59.023584771Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_tAf18T7A-WGzT8ffKm9GGgu","ver":1},"status":{"status":"write"}}},{"id":"1_1615423619.028138_99.1","section":"bucket.instance","name":"nolznk-9:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.13","timestamp":"2021-03-11T00:46:59.028138621Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_tAf18T7A-WGzT8ffKm9GGgu","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423619.029573_100.1","section":"bucket","name":"nolznk-9","timestamp":"2021-03-11T00:46:59.029573417Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Yu95E0IeKnKtPiXbPggWQ80","ver":1},"status":{"status":"write"}}},{"id":"1_1615423619.033309_101.1","section":"bucket","name":"nolznk-9","timestamp":"2021-03-11T00:46:59.033309932Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Yu95E0IeKnKtPiXbPggWQ80","ver":1},"status":{"status":"complete"}}}] 2021-03-11T00:51:53.714 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '60afeec2-94d4-4f5a-a646-d47261804cd8', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:53.714 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '60afeec2-94d4-4f5a-a646-d47261804cd8', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:53.715 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 60afeec2-94d4-4f5a-a646-d47261804cd8 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:51:53.755 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:51:53.756 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:51:53.949 INFO:teuthology.orchestra.run.smithi119.stdout:[{"id":"1_1615423843.334639_252.1","section":"bucket.instance","name":"nolznk-29:a2f6d1db-e7f6-4c80-8390-898891b9eb91.5161.1","timestamp":"2021-03-11T00:50:43.334639030Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rZUhX8tN5VU5JNjMyM844Yy","ver":1},"status":{"status":"write"}}},{"id":"1_1615423843.338639_253.1","section":"bucket.instance","name":"nolznk-29:a2f6d1db-e7f6-4c80-8390-898891b9eb91.5161.1","timestamp":"2021-03-11T00:50:43.338639375Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rZUhX8tN5VU5JNjMyM844Yy","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423843.339973_254.1","section":"bucket","name":"nolznk-29","timestamp":"2021-03-11T00:50:43.339973737Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_lJR_4NK6HcLhxM3kAN0MW7J","ver":1},"status":{"status":"write"}}},{"id":"1_1615423843.343428_255.1","section":"bucket","name":"nolznk-29","timestamp":"2021-03-11T00:50:43.343428761Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_lJR_4NK6HcLhxM3kAN0MW7J","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423843.380369_257.1","section":"bucket.instance","name":"nolznk-30:a2f6d1db-e7f6-4c80-8390-898891b9eb91.5161.2","timestamp":"2021-03-11T00:50:43.380369410Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_P445-OF1iAGR_SGBxaeLOch","ver":1},"status":{"status":"write"}}},{"id":"1_1615423843.386196_258.1","section":"bucket.instance","name":"nolznk-30:a2f6d1db-e7f6-4c80-8390-898891b9eb91.5161.2","timestamp":"2021-03-11T00:50:43.386196316Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_P445-OF1iAGR_SGBxaeLOch","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423843.388282_259.1","section":"bucket","name":"nolznk-30","timestamp":"2021-03-11T00:50:43.388282791Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_59vDkmriKFflFi7r-HsDOnx","ver":1},"status":{"status":"write"}}},{"id":"1_1615423843.392949_260.1","section":"bucket","name":"nolznk-30","timestamp":"2021-03-11T00:50:43.392949926Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_59vDkmriKFflFi7r-HsDOnx","ver":1},"status":{"status":"complete"}}}] 2021-03-11T00:51:53.950 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '60afeec2-94d4-4f5a-a646-d47261804cd8', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:53.950 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '60afeec2-94d4-4f5a-a646-d47261804cd8', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:53.951 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 60afeec2-94d4-4f5a-a646-d47261804cd8 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:51:53.989 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:51:53.990 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:51:54.167 INFO:teuthology.orchestra.run.smithi124.stdout:[{"id":"1_1615423843.334639_252.1","section":"bucket.instance","name":"nolznk-29:a2f6d1db-e7f6-4c80-8390-898891b9eb91.5161.1","timestamp":"2021-03-11T00:50:43.334639030Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rZUhX8tN5VU5JNjMyM844Yy","ver":1},"status":{"status":"write"}}},{"id":"1_1615423843.338639_253.1","section":"bucket.instance","name":"nolznk-29:a2f6d1db-e7f6-4c80-8390-898891b9eb91.5161.1","timestamp":"2021-03-11T00:50:43.338639375Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rZUhX8tN5VU5JNjMyM844Yy","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423843.339973_254.1","section":"bucket","name":"nolznk-29","timestamp":"2021-03-11T00:50:43.339973737Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_lJR_4NK6HcLhxM3kAN0MW7J","ver":1},"status":{"status":"write"}}},{"id":"1_1615423843.343428_255.1","section":"bucket","name":"nolznk-29","timestamp":"2021-03-11T00:50:43.343428761Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_lJR_4NK6HcLhxM3kAN0MW7J","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423843.380369_257.1","section":"bucket.instance","name":"nolznk-30:a2f6d1db-e7f6-4c80-8390-898891b9eb91.5161.2","timestamp":"2021-03-11T00:50:43.380369410Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_P445-OF1iAGR_SGBxaeLOch","ver":1},"status":{"status":"write"}}},{"id":"1_1615423843.386196_258.1","section":"bucket.instance","name":"nolznk-30:a2f6d1db-e7f6-4c80-8390-898891b9eb91.5161.2","timestamp":"2021-03-11T00:50:43.386196316Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_P445-OF1iAGR_SGBxaeLOch","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423843.388282_259.1","section":"bucket","name":"nolznk-30","timestamp":"2021-03-11T00:50:43.388282791Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_59vDkmriKFflFi7r-HsDOnx","ver":1},"status":{"status":"write"}}},{"id":"1_1615423843.392949_260.1","section":"bucket","name":"nolznk-30","timestamp":"2021-03-11T00:50:43.392949926Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_59vDkmriKFflFi7r-HsDOnx","ver":1},"status":{"status":"complete"}}}] 2021-03-11T00:51:54.168 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '60afeec2-94d4-4f5a-a646-d47261804cd8', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:54.168 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '60afeec2-94d4-4f5a-a646-d47261804cd8', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:54.169 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 60afeec2-94d4-4f5a-a646-d47261804cd8 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:51:54.254 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:51:54.255 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:51:54.432 INFO:teuthology.orchestra.run.smithi124.stdout:[{"id":"1_1615423843.334639_252.1","section":"bucket.instance","name":"nolznk-29:a2f6d1db-e7f6-4c80-8390-898891b9eb91.5161.1","timestamp":"2021-03-11T00:50:43.334639030Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rZUhX8tN5VU5JNjMyM844Yy","ver":1},"status":{"status":"write"}}},{"id":"1_1615423843.338639_253.1","section":"bucket.instance","name":"nolznk-29:a2f6d1db-e7f6-4c80-8390-898891b9eb91.5161.1","timestamp":"2021-03-11T00:50:43.338639375Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_rZUhX8tN5VU5JNjMyM844Yy","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423843.339973_254.1","section":"bucket","name":"nolznk-29","timestamp":"2021-03-11T00:50:43.339973737Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_lJR_4NK6HcLhxM3kAN0MW7J","ver":1},"status":{"status":"write"}}},{"id":"1_1615423843.343428_255.1","section":"bucket","name":"nolznk-29","timestamp":"2021-03-11T00:50:43.343428761Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_lJR_4NK6HcLhxM3kAN0MW7J","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423843.380369_257.1","section":"bucket.instance","name":"nolznk-30:a2f6d1db-e7f6-4c80-8390-898891b9eb91.5161.2","timestamp":"2021-03-11T00:50:43.380369410Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_P445-OF1iAGR_SGBxaeLOch","ver":1},"status":{"status":"write"}}},{"id":"1_1615423843.386196_258.1","section":"bucket.instance","name":"nolznk-30:a2f6d1db-e7f6-4c80-8390-898891b9eb91.5161.2","timestamp":"2021-03-11T00:50:43.386196316Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_P445-OF1iAGR_SGBxaeLOch","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423843.388282_259.1","section":"bucket","name":"nolznk-30","timestamp":"2021-03-11T00:50:43.388282791Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_59vDkmriKFflFi7r-HsDOnx","ver":1},"status":{"status":"write"}}},{"id":"1_1615423843.392949_260.1","section":"bucket","name":"nolznk-30","timestamp":"2021-03-11T00:50:43.392949926Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_59vDkmriKFflFi7r-HsDOnx","ver":1},"status":{"status":"complete"}}}] 2021-03-11T00:51:54.433 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '76343c7e-8958-4b2a-a007-ca994322980a', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:54.433 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '76343c7e-8958-4b2a-a007-ca994322980a', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:54.434 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 76343c7e-8958-4b2a-a007-ca994322980a --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:51:54.474 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:51:54.475 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:51:54.664 INFO:teuthology.orchestra.run.smithi119.stdout:[{"id":"1_1615423909.541811_70.1","section":"bucket.instance","name":"nolznk-31:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4757.1","timestamp":"2021-03-11T00:51:49.541811568Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_kDHJxrPd9PG-rCYVpYy4Fuj","ver":1},"status":{"status":"write"}}},{"id":"1_1615423909.546379_71.1","section":"bucket.instance","name":"nolznk-31:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4757.1","timestamp":"2021-03-11T00:51:49.546379093Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_kDHJxrPd9PG-rCYVpYy4Fuj","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423909.547844_72.1","section":"bucket","name":"nolznk-31","timestamp":"2021-03-11T00:51:49.547844191Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VEi4umDJHxu0Pwl7495a18z","ver":1},"status":{"status":"write"}}},{"id":"1_1615423909.551275_73.1","section":"bucket","name":"nolznk-31","timestamp":"2021-03-11T00:51:49.551275622Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VEi4umDJHxu0Pwl7495a18z","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423909.572272_74.1","section":"bucket.instance","name":"nolznk-32:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4757.2","timestamp":"2021-03-11T00:51:49.572272937Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_21n1VFv590zlxGORhUgEtfx","ver":1},"status":{"status":"write"}}},{"id":"1_1615423909.577496_75.1","section":"bucket.instance","name":"nolznk-32:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4757.2","timestamp":"2021-03-11T00:51:49.577496816Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_21n1VFv590zlxGORhUgEtfx","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423909.579263_76.1","section":"bucket","name":"nolznk-32","timestamp":"2021-03-11T00:51:49.579263427Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ml8QmEY_x9WpWn1Xyu8qZjN","ver":1},"status":{"status":"write"}}},{"id":"1_1615423909.583891_77.1","section":"bucket","name":"nolznk-32","timestamp":"2021-03-11T00:51:49.583891625Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ml8QmEY_x9WpWn1Xyu8qZjN","ver":1},"status":{"status":"complete"}}}] 2021-03-11T00:51:54.664 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '76343c7e-8958-4b2a-a007-ca994322980a', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:54.665 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '76343c7e-8958-4b2a-a007-ca994322980a', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:54.665 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 76343c7e-8958-4b2a-a007-ca994322980a --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:51:54.707 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:51:54.708 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:51:54.872 INFO:teuthology.orchestra.run.smithi124.stdout:[{"id":"1_1615423909.541811_70.1","section":"bucket.instance","name":"nolznk-31:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4757.1","timestamp":"2021-03-11T00:51:49.541811568Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_kDHJxrPd9PG-rCYVpYy4Fuj","ver":1},"status":{"status":"write"}}},{"id":"1_1615423909.546379_71.1","section":"bucket.instance","name":"nolznk-31:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4757.1","timestamp":"2021-03-11T00:51:49.546379093Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_kDHJxrPd9PG-rCYVpYy4Fuj","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423909.547844_72.1","section":"bucket","name":"nolznk-31","timestamp":"2021-03-11T00:51:49.547844191Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VEi4umDJHxu0Pwl7495a18z","ver":1},"status":{"status":"write"}}},{"id":"1_1615423909.551275_73.1","section":"bucket","name":"nolznk-31","timestamp":"2021-03-11T00:51:49.551275622Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VEi4umDJHxu0Pwl7495a18z","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423909.572272_74.1","section":"bucket.instance","name":"nolznk-32:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4757.2","timestamp":"2021-03-11T00:51:49.572272937Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_21n1VFv590zlxGORhUgEtfx","ver":1},"status":{"status":"write"}}},{"id":"1_1615423909.577496_75.1","section":"bucket.instance","name":"nolznk-32:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4757.2","timestamp":"2021-03-11T00:51:49.577496816Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_21n1VFv590zlxGORhUgEtfx","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423909.579263_76.1","section":"bucket","name":"nolznk-32","timestamp":"2021-03-11T00:51:49.579263427Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ml8QmEY_x9WpWn1Xyu8qZjN","ver":1},"status":{"status":"write"}}},{"id":"1_1615423909.583891_77.1","section":"bucket","name":"nolznk-32","timestamp":"2021-03-11T00:51:49.583891625Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ml8QmEY_x9WpWn1Xyu8qZjN","ver":1},"status":{"status":"complete"}}}] 2021-03-11T00:51:54.872 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '76343c7e-8958-4b2a-a007-ca994322980a', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:54.873 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '76343c7e-8958-4b2a-a007-ca994322980a', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:54.874 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 76343c7e-8958-4b2a-a007-ca994322980a --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:51:54.953 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:51:54.954 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:51:55.120 INFO:teuthology.orchestra.run.smithi124.stdout:[{"id":"1_1615423909.541811_70.1","section":"bucket.instance","name":"nolznk-31:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4757.1","timestamp":"2021-03-11T00:51:49.541811568Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_kDHJxrPd9PG-rCYVpYy4Fuj","ver":1},"status":{"status":"write"}}},{"id":"1_1615423909.546379_71.1","section":"bucket.instance","name":"nolznk-31:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4757.1","timestamp":"2021-03-11T00:51:49.546379093Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_kDHJxrPd9PG-rCYVpYy4Fuj","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423909.547844_72.1","section":"bucket","name":"nolznk-31","timestamp":"2021-03-11T00:51:49.547844191Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VEi4umDJHxu0Pwl7495a18z","ver":1},"status":{"status":"write"}}},{"id":"1_1615423909.551275_73.1","section":"bucket","name":"nolznk-31","timestamp":"2021-03-11T00:51:49.551275622Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VEi4umDJHxu0Pwl7495a18z","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423909.572272_74.1","section":"bucket.instance","name":"nolznk-32:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4757.2","timestamp":"2021-03-11T00:51:49.572272937Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_21n1VFv590zlxGORhUgEtfx","ver":1},"status":{"status":"write"}}},{"id":"1_1615423909.577496_75.1","section":"bucket.instance","name":"nolznk-32:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4757.2","timestamp":"2021-03-11T00:51:49.577496816Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_21n1VFv590zlxGORhUgEtfx","ver":1},"status":{"status":"complete"}}},{"id":"1_1615423909.579263_76.1","section":"bucket","name":"nolznk-32","timestamp":"2021-03-11T00:51:49.579263427Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ml8QmEY_x9WpWn1Xyu8qZjN","ver":1},"status":{"status":"write"}}},{"id":"1_1615423909.583891_77.1","section":"bucket","name":"nolznk-32","timestamp":"2021-03-11T00:51:49.583891625Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ml8QmEY_x9WpWn1Xyu8qZjN","ver":1},"status":{"status":"complete"}}}] 2021-03-11T00:51:55.121 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:51:55.122 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:51:55.122 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog autotrim --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T00:51:55.158 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:51:55.159 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:51:55.395 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'autotrim', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:51:55.396 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'autotrim', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:51:55.397 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog autotrim --cluster c2 --debug-rgw 0 --debug-ms 0 2021-03-11T00:51:55.434 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:51:55.435 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:51:55.787 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'autotrim', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:51:55.788 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'autotrim', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:51:55.789 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog autotrim --cluster c2 --debug-rgw 0 --debug-ms 0 2021-03-11T00:51:55.864 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:51:55.865 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:51:56.202 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '470410ef-8c66-4339-a004-064d3e2c6c91', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:56.203 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '470410ef-8c66-4339-a004-064d3e2c6c91', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:56.204 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 470410ef-8c66-4339-a004-064d3e2c6c91 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:51:56.244 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:51:56.245 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:51:56.438 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2021-03-11T00:51:56.695 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '470410ef-8c66-4339-a004-064d3e2c6c91', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:56.696 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '470410ef-8c66-4339-a004-064d3e2c6c91', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:56.696 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 470410ef-8c66-4339-a004-064d3e2c6c91 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:51:56.740 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:51:56.740 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:51:56.932 INFO:teuthology.orchestra.run.smithi124.stdout:[] 2021-03-11T00:51:56.933 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '470410ef-8c66-4339-a004-064d3e2c6c91', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:56.934 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '470410ef-8c66-4339-a004-064d3e2c6c91', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:56.934 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 470410ef-8c66-4339-a004-064d3e2c6c91 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:51:57.014 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:51:57.015 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:51:57.197 INFO:teuthology.orchestra.run.smithi124.stdout:[] 2021-03-11T00:51:57.198 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '60afeec2-94d4-4f5a-a646-d47261804cd8', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:57.198 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '60afeec2-94d4-4f5a-a646-d47261804cd8', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:57.199 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 60afeec2-94d4-4f5a-a646-d47261804cd8 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:51:57.239 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:51:57.240 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:51:57.426 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2021-03-11T00:51:57.426 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '60afeec2-94d4-4f5a-a646-d47261804cd8', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:57.427 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '60afeec2-94d4-4f5a-a646-d47261804cd8', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:57.428 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 60afeec2-94d4-4f5a-a646-d47261804cd8 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:51:57.466 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:51:57.466 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:51:57.644 INFO:teuthology.orchestra.run.smithi124.stdout:[] 2021-03-11T00:51:57.645 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '60afeec2-94d4-4f5a-a646-d47261804cd8', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:57.645 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '60afeec2-94d4-4f5a-a646-d47261804cd8', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:57.646 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 60afeec2-94d4-4f5a-a646-d47261804cd8 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:51:57.725 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:51:57.726 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:51:57.909 INFO:teuthology.orchestra.run.smithi124.stdout:[] 2021-03-11T00:51:57.909 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '76343c7e-8958-4b2a-a007-ca994322980a', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:57.910 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '76343c7e-8958-4b2a-a007-ca994322980a', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:57.911 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 76343c7e-8958-4b2a-a007-ca994322980a --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:51:57.951 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:51:57.951 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:51:58.155 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2021-03-11T00:51:58.155 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '76343c7e-8958-4b2a-a007-ca994322980a', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:58.156 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '76343c7e-8958-4b2a-a007-ca994322980a', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:58.156 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 76343c7e-8958-4b2a-a007-ca994322980a --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:51:58.196 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:51:58.197 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:51:58.395 INFO:teuthology.orchestra.run.smithi124.stdout:[] 2021-03-11T00:51:58.396 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '76343c7e-8958-4b2a-a007-ca994322980a', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:58.396 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '76343c7e-8958-4b2a-a007-ca994322980a', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:58.397 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 76343c7e-8958-4b2a-a007-ca994322980a --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:51:58.479 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:51:58.479 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:51:58.675 INFO:teuthology.orchestra.run.smithi124.stdout:[] 2021-03-11T00:51:58.676 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_multi_period_incremental_sync ... ok 2021-03-11T00:51:58.679 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=nolznk-33 2021-03-11T00:51:58.707 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=nolznk-34 2021-03-11T00:51:58.851 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:58.851 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:58.852 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:51:58.893 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:51:58.893 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:51:59.075 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615423685.919484_54.3","last_update":"2021-03-11T00:48:05.919484Z"},{"marker":"1_1615423777.100275_264.1","last_update":"2021-03-11T00:49:37.100275Z"},{"marker":"1_1615423685.919484_83.2","last_update":"2021-03-11T00:48:05.919484Z"},{"marker":"1_1615423918.765365_200.1","last_update":"2021-03-11T00:51:58.765365Z"}] 2021-03-11T00:51:59.076 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2021-03-11T00:51:59.076 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:59.077 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:51:59.078 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:51:59.119 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:51:59.119 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:51:59.306 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17085014794446772242},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_54.3","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423777.100275_264.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:49:37.100275Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_83.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423773.936408_152.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:49:33.936408Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:51:59.306 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone1: shard=3 master=1_1615423918.765365_200.1 target=1_1615423773.936408_152.2 2021-03-11T00:51:59.802 INFO:tasks.rgw.c2.client.0.smithi124.stdout:2021-03-11T00:51:59.796+0000 7fd2b27f4700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T00:51:59.803 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T00:51:59.796+0000 7fcc777fe700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T00:51:59.805 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T00:51:59.800+0000 7fcc6ffff700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T00:51:59.808 INFO:tasks.rgw.c2.client.0.smithi124.stdout:2021-03-11T00:51:59.800+0000 7fd2ac7e8700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T00:52:04.308 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:52:04.309 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:52:04.309 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:52:04.353 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:52:04.354 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:52:04.537 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17085014794446772242},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_54.3","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423777.100275_264.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:49:37.100275Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_83.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423920.063551_209.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:52:00.063551Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:52:04.538 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2021-03-11T00:52:04.538 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:52:04.539 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:52:04.539 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:52:04.580 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:52:04.580 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:52:04.768 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615423685.919484_54.3","last_update":"2021-03-11T00:48:05.919484Z"},{"marker":"1_1615423777.100275_264.1","last_update":"2021-03-11T00:49:37.100275Z"},{"marker":"1_1615423685.919484_83.2","last_update":"2021-03-11T00:48:05.919484Z"},{"marker":"1_1615423920.063551_209.1","last_update":"2021-03-11T00:52:00.063551Z"}] 2021-03-11T00:52:04.769 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone3 source_zone=test-zone1 2021-03-11T00:52:04.769 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:52:04.770 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:52:04.771 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:52:04.815 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:52:04.815 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:52:05.071 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":6834771546787753244},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_54.3","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423777.100275_264.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:49:37.100275Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_83.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423920.063551_209.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:52:00.063551Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:52:05.072 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone3 source_zone=test-zone1 2021-03-11T00:52:05.073 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:52:05.073 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:52:05.074 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:52:05.153 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:52:05.154 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:52:05.346 INFO:teuthology.orchestra.run.smithi124.stdout:[{"marker":"1_1615423686.841255_28.3","last_update":"2021-03-11T00:48:06.841255Z"},{"marker":"1_1615423776.997988_162.1","last_update":"2021-03-11T00:49:36.997988Z"},{"marker":"1_1615423686.841255_82.2","last_update":"2021-03-11T00:48:06.841255Z"},{"marker":"1_1615423919.977312_197.1","last_update":"2021-03-11T00:51:59.977312Z"}] 2021-03-11T00:52:05.347 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2021-03-11T00:52:05.347 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:52:05.348 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:52:05.348 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:52:05.392 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:52:05.393 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:52:05.578 INFO:teuthology.orchestra.run.smithi119.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":3384081757615347012},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615423686.841255_28.3","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:06.841255Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423776.997988_162.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:49:36.997988Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423686.841255_82.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:06.841255Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423919.977312_197.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:51:59.977312Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:52:05.579 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2021-03-11T00:52:05.579 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:52:05.580 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:52:05.580 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog autotrim --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T00:52:05.660 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:52:05.661 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:52:05.844 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'list', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:52:05.845 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'list', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:52:05.846 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog list --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:52:05.922 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:52:05.922 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:52:06.128 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2021-03-11T00:52:06.129 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'autotrim', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:52:06.129 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'autotrim', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:52:06.130 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog autotrim --cluster c2 --debug-rgw 0 --debug-ms 0 2021-03-11T00:52:06.167 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:52:06.167 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:52:06.364 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'list', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:52:06.365 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'list', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:52:06.365 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog list --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:52:06.443 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:52:06.444 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:52:06.630 INFO:teuthology.orchestra.run.smithi124.stdout:[] 2021-03-11T00:52:06.631 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_datalog_autotrim ... ok 2021-03-11T00:52:06.632 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--sync-from-all=false', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:52:06.634 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--sync-from-all=false', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:52:06.635 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone modify --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --sync-from-all=false --cluster c2 --debug-rgw 0 --debug-ms 0 2021-03-11T00:52:06.711 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:52:06.711 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:52:06.737 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:52:06.732+0000 7f7bd0688080 0 failed reading obj info from .rgw.root:zone_info.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0: (2) No such file or directory 2021-03-11T00:52:06.738 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:52:06.732+0000 7f7bd0688080 0 WARNING: could not read zone params for zone id=4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0 name=test-zone1 2021-03-11T00:52:06.749 INFO:teuthology.orchestra.run.smithi124.stdout:{"id":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","name":"test-zone2","domain_root":"test-zone2.rgw.meta:root","control_pool":"test-zone2.rgw.control","gc_pool":"test-zone2.rgw.log:gc","lc_pool":"test-zone2.rgw.log:lc","log_pool":"test-zone2.rgw.log","intent_log_pool":"test-zone2.rgw.log:intent","usage_log_pool":"test-zone2.rgw.log:usage","roles_pool":"test-zone2.rgw.meta:roles","reshard_pool":"test-zone2.rgw.log:reshard","user_keys_pool":"test-zone2.rgw.meta:users.keys","user_email_pool":"test-zone2.rgw.meta:users.email","user_swift_pool":"test-zone2.rgw.meta:users.swift","user_uid_pool":"test-zone2.rgw.meta:users.uid","otp_pool":"test-zone2.rgw.otp","system_key":{"access_key":"8WRXV4QK6AZ20SFM","secret_key":"q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone2.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone2.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone2.rgw.buckets.non-ec","index_type":0}}],"realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5"} 2021-03-11T00:52:06.750 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:52:06.750 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:52:06.751 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2021-03-11T00:52:06.828 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:52:06.829 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:52:07.016 INFO:teuthology.orchestra.run.smithi124.stderr:Sending period to new master zone 4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0 2021-03-11T00:52:08.103 INFO:teuthology.orchestra.run.smithi124.stdout:{"id":"76343c7e-8958-4b2a-a007-ca994322980a","epoch":2,"predecessor_uuid":"60afeec2-94d4-4f5a-a646-d47261804cd8","sync_status":["1_1615423843.392949_260.1","","",""],"period_map":{"id":"76343c7e-8958-4b2a-a007-ca994322980a","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","zones":[{"id":"2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05","name":"test-zone3","endpoints":["http://smithi124.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","name":"test-zone1","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","name":"test-zone2","endpoints":["http://smithi124.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"false","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","sync_policy":{"groups":[]}}],"short_zone_ids":[{"key":"2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05","val":906122584},{"key":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","val":3593284871},{"key":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","val":1094358664}]},"master_zonegroup":"test-zonegroup","master_zone":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","realm_name":"test-realm","realm_epoch":4} 2021-03-11T00:52:08.104 INFO:rgw_multi.tests:Set sync_from_all flag on zone test-zone2 to false 2021-03-11T00:53:08.105 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=nolznk-35 2021-03-11T00:53:08.195 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:53:08.196 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:53:08.197 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:53:08.237 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:53:08.237 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:53:08.395 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=76343c7e-8958-4b2a-a007-ca994322980a 2021-03-11T00:53:08.426 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615423988.143025_100.1","last_update":"2021-03-11T00:53:08.143025Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T00:53:08.427 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2021-03-11T00:53:08.428 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:53:08.428 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:53:08.429 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:53:08.469 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:53:08.470 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:53:08.651 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423918.729994_86.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:51:58.729994Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:53:08.652 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423988.143025_100.1 target=1_1615423918.729994_86.1 2021-03-11T00:53:13.654 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:53:13.661 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:53:13.662 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:53:13.705 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:53:13.705 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:53:13.884 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423918.729994_86.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:51:58.729994Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:53:13.885 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423988.143025_100.1 target=1_1615423918.729994_86.1 2021-03-11T00:53:18.887 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:53:18.893 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:53:18.894 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:53:18.935 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:53:18.936 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:53:19.123 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423918.729994_86.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:51:58.729994Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:53:19.124 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423988.143025_100.1 target=1_1615423918.729994_86.1 2021-03-11T00:53:24.125 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:53:24.126 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:53:24.127 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:53:24.169 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:53:24.169 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:53:24.357 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423918.729994_86.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:51:58.729994Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:53:24.358 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423988.143025_100.1 target=1_1615423918.729994_86.1 2021-03-11T00:53:29.359 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:53:29.360 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:53:29.360 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:53:29.412 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:53:29.412 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:53:29.595 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423918.729994_86.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:51:58.729994Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:53:29.595 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423988.143025_100.1 target=1_1615423918.729994_86.1 2021-03-11T00:53:34.596 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:53:34.597 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:53:34.598 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:53:34.637 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:53:34.638 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:53:34.828 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423918.729994_86.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:51:58.729994Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:53:34.829 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615423988.143025_100.1 target=1_1615423918.729994_86.1 2021-03-11T00:53:39.830 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:53:39.831 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:53:39.832 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:53:39.874 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:53:39.874 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:53:40.055 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423988.143025_100.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:53:08.143025Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:53:40.056 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2021-03-11T00:53:40.057 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T00:53:40.057 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:53:40.058 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:53:40.058 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:53:40.141 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:53:40.142 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:53:40.398 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423918.729994_86.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:51:58.729994Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:53:40.399 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1615423988.143025_100.1 target=1_1615423918.729994_86.1 2021-03-11T00:53:45.400 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:53:45.401 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:53:45.402 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:53:45.441 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:53:45.441 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:53:45.799 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423918.729994_86.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:51:58.729994Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:53:45.799 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1615423988.143025_100.1 target=1_1615423918.729994_86.1 2021-03-11T00:53:50.801 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:53:50.802 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:53:50.802 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:53:50.850 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:53:50.850 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:53:51.107 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423918.729994_86.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:51:58.729994Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:53:51.107 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1615423988.143025_100.1 target=1_1615423918.729994_86.1 2021-03-11T00:53:56.109 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:53:56.110 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:53:56.110 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:53:56.151 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:53:56.151 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:53:56.408 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423918.729994_86.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:51:58.729994Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:53:56.409 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1615423988.143025_100.1 target=1_1615423918.729994_86.1 2021-03-11T00:54:01.410 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:54:01.411 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:54:01.412 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:54:01.454 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:54:01.455 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:54:01.713 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423918.729994_86.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:51:58.729994Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:54:01.714 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1615423988.143025_100.1 target=1_1615423918.729994_86.1 2021-03-11T00:54:06.716 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:54:06.724 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:54:06.725 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:54:06.765 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:54:06.766 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:54:07.120 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423918.729994_86.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:51:58.729994Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:54:07.120 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1615423988.143025_100.1 target=1_1615423918.729994_86.1 2021-03-11T00:54:09.240 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T00:54:09.230+0000 7fcc6cff9700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T00:54:12.121 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:54:12.127 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:54:12.128 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:54:12.169 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:54:12.170 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:54:12.425 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423988.143025_100.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:53:08.143025Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:54:12.426 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T00:54:12.441 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--redirect-zone=4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:54:12.451 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--redirect-zone=4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:54:12.451 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone modify --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --redirect-zone=4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0 --cluster c2 --debug-rgw 0 --debug-ms 0 2021-03-11T00:54:12.508 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:54:12.508 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:54:12.531 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:54:12.522+0000 7f0d1eddf080 0 failed reading obj info from .rgw.root:zone_info.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0: (2) No such file or directory 2021-03-11T00:54:12.531 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:54:12.522+0000 7f0d1eddf080 0 WARNING: could not read zone params for zone id=4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0 name=test-zone1 2021-03-11T00:54:12.541 INFO:teuthology.orchestra.run.smithi124.stdout:{"id":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","name":"test-zone2","domain_root":"test-zone2.rgw.meta:root","control_pool":"test-zone2.rgw.control","gc_pool":"test-zone2.rgw.log:gc","lc_pool":"test-zone2.rgw.log:lc","log_pool":"test-zone2.rgw.log","intent_log_pool":"test-zone2.rgw.log:intent","usage_log_pool":"test-zone2.rgw.log:usage","roles_pool":"test-zone2.rgw.meta:roles","reshard_pool":"test-zone2.rgw.log:reshard","user_keys_pool":"test-zone2.rgw.meta:users.keys","user_email_pool":"test-zone2.rgw.meta:users.email","user_swift_pool":"test-zone2.rgw.meta:users.swift","user_uid_pool":"test-zone2.rgw.meta:users.uid","otp_pool":"test-zone2.rgw.otp","system_key":{"access_key":"8WRXV4QK6AZ20SFM","secret_key":"q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone2.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone2.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone2.rgw.buckets.non-ec","index_type":0}}],"realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5"} 2021-03-11T00:54:12.542 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:54:12.542 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:54:12.543 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2021-03-11T00:54:12.623 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:54:12.623 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:54:12.814 INFO:teuthology.orchestra.run.smithi124.stderr:Sending period to new master zone 4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0 2021-03-11T00:54:13.900 INFO:teuthology.orchestra.run.smithi124.stdout:{"id":"76343c7e-8958-4b2a-a007-ca994322980a","epoch":3,"predecessor_uuid":"60afeec2-94d4-4f5a-a646-d47261804cd8","sync_status":["1_1615423843.392949_260.1","","",""],"period_map":{"id":"76343c7e-8958-4b2a-a007-ca994322980a","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","zones":[{"id":"2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05","name":"test-zone3","endpoints":["http://smithi124.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","name":"test-zone1","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","name":"test-zone2","endpoints":["http://smithi124.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"false","sync_from":[],"redirect_zone":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0"}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","sync_policy":{"groups":[]}}],"short_zone_ids":[{"key":"2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05","val":906122584},{"key":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","val":3593284871},{"key":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","val":1094358664}]},"master_zonegroup":"test-zonegroup","master_zone":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","realm_name":"test-realm","realm_epoch":4} 2021-03-11T00:54:13.901 INFO:rgw_multi.tests:Set redirect_zone zone test-zone2 to "4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0" 2021-03-11T00:55:14.226 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--sync-from-all=true', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:55:14.228 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--sync-from-all=true', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:55:14.229 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone modify --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --sync-from-all=true --cluster c2 --debug-rgw 0 --debug-ms 0 2021-03-11T00:55:14.268 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:55:14.268 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:55:14.296 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:55:14.292+0000 7f68d0b60080 0 failed reading obj info from .rgw.root:zone_info.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0: (2) No such file or directory 2021-03-11T00:55:14.296 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:55:14.292+0000 7f68d0b60080 0 WARNING: could not read zone params for zone id=4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0 name=test-zone1 2021-03-11T00:55:14.308 INFO:teuthology.orchestra.run.smithi124.stdout:{"id":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","name":"test-zone2","domain_root":"test-zone2.rgw.meta:root","control_pool":"test-zone2.rgw.control","gc_pool":"test-zone2.rgw.log:gc","lc_pool":"test-zone2.rgw.log:lc","log_pool":"test-zone2.rgw.log","intent_log_pool":"test-zone2.rgw.log:intent","usage_log_pool":"test-zone2.rgw.log:usage","roles_pool":"test-zone2.rgw.meta:roles","reshard_pool":"test-zone2.rgw.log:reshard","user_keys_pool":"test-zone2.rgw.meta:users.keys","user_email_pool":"test-zone2.rgw.meta:users.email","user_swift_pool":"test-zone2.rgw.meta:users.swift","user_uid_pool":"test-zone2.rgw.meta:users.uid","otp_pool":"test-zone2.rgw.otp","system_key":{"access_key":"8WRXV4QK6AZ20SFM","secret_key":"q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone2.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone2.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone2.rgw.buckets.non-ec","index_type":0}}],"realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5"} 2021-03-11T00:55:14.308 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:55:14.309 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:55:14.309 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2021-03-11T00:55:14.394 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:55:14.395 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:55:14.568 INFO:teuthology.orchestra.run.smithi124.stderr:Sending period to new master zone 4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0 2021-03-11T00:55:15.654 INFO:teuthology.orchestra.run.smithi124.stdout:{"id":"76343c7e-8958-4b2a-a007-ca994322980a","epoch":4,"predecessor_uuid":"60afeec2-94d4-4f5a-a646-d47261804cd8","sync_status":["1_1615423843.392949_260.1","","",""],"period_map":{"id":"76343c7e-8958-4b2a-a007-ca994322980a","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","zones":[{"id":"2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05","name":"test-zone3","endpoints":["http://smithi124.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","name":"test-zone1","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","name":"test-zone2","endpoints":["http://smithi124.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0"}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","sync_policy":{"groups":[]}}],"short_zone_ids":[{"key":"2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05","val":906122584},{"key":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","val":3593284871},{"key":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","val":1094358664}]},"master_zonegroup":"test-zonegroup","master_zone":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","realm_name":"test-realm","realm_epoch":4} 2021-03-11T00:55:15.654 INFO:rgw_multi.tests:Set sync_from_all flag on zone test-zone2 to true 2021-03-11T00:56:15.656 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--redirect-zone=', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:56:15.672 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--redirect-zone=', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:56:15.674 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone modify --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --redirect-zone= --cluster c2 --debug-rgw 0 --debug-ms 0 2021-03-11T00:56:15.714 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:56:15.715 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:56:15.740 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:56:15.731+0000 7f81d6fba080 0 failed reading obj info from .rgw.root:zone_info.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0: (2) No such file or directory 2021-03-11T00:56:15.740 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:56:15.735+0000 7f81d6fba080 0 WARNING: could not read zone params for zone id=4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0 name=test-zone1 2021-03-11T00:56:15.752 INFO:teuthology.orchestra.run.smithi124.stdout:{"id":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","name":"test-zone2","domain_root":"test-zone2.rgw.meta:root","control_pool":"test-zone2.rgw.control","gc_pool":"test-zone2.rgw.log:gc","lc_pool":"test-zone2.rgw.log:lc","log_pool":"test-zone2.rgw.log","intent_log_pool":"test-zone2.rgw.log:intent","usage_log_pool":"test-zone2.rgw.log:usage","roles_pool":"test-zone2.rgw.meta:roles","reshard_pool":"test-zone2.rgw.log:reshard","user_keys_pool":"test-zone2.rgw.meta:users.keys","user_email_pool":"test-zone2.rgw.meta:users.email","user_swift_pool":"test-zone2.rgw.meta:users.swift","user_uid_pool":"test-zone2.rgw.meta:users.uid","otp_pool":"test-zone2.rgw.otp","system_key":{"access_key":"8WRXV4QK6AZ20SFM","secret_key":"q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone2.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone2.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone2.rgw.buckets.non-ec","index_type":0}}],"realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5"} 2021-03-11T00:56:15.752 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:56:15.753 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:56:15.754 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2021-03-11T00:56:15.835 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:56:15.835 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:56:16.020 INFO:teuthology.orchestra.run.smithi124.stderr:Sending period to new master zone 4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0 2021-03-11T00:56:17.106 INFO:teuthology.orchestra.run.smithi124.stdout:{"id":"76343c7e-8958-4b2a-a007-ca994322980a","epoch":5,"predecessor_uuid":"60afeec2-94d4-4f5a-a646-d47261804cd8","sync_status":["1_1615423843.392949_260.1","","",""],"period_map":{"id":"76343c7e-8958-4b2a-a007-ca994322980a","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","zones":[{"id":"2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05","name":"test-zone3","endpoints":["http://smithi124.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","name":"test-zone1","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","name":"test-zone2","endpoints":["http://smithi124.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","sync_policy":{"groups":[]}}],"short_zone_ids":[{"key":"2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05","val":906122584},{"key":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","val":3593284871},{"key":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","val":1094358664}]},"master_zonegroup":"test-zonegroup","master_zone":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","realm_name":"test-realm","realm_epoch":4} 2021-03-11T00:56:17.107 INFO:rgw_multi.tests:Set redirect_zone zone test-zone2 to "" 2021-03-11T00:57:17.109 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_multi_zone_redirect ... ok 2021-03-11T00:57:17.111 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:57:17.111 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:57:17.112 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:57:17.151 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:57:17.151 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:57:17.305 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=76343c7e-8958-4b2a-a007-ca994322980a 2021-03-11T00:57:17.334 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615423988.143025_100.1","last_update":"2021-03-11T00:53:08.143025Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T00:57:17.334 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2021-03-11T00:57:17.335 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:57:17.335 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:57:17.336 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:57:17.377 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:57:17.377 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:57:17.585 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423988.143025_100.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:53:08.143025Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:57:17.586 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2021-03-11T00:57:17.586 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T00:57:17.587 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:57:17.587 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:57:17.588 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:57:17.665 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:57:17.666 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:57:17.921 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615423988.143025_100.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T00:53:08.143025Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T00:57:17.922 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T00:57:17.923 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'create', '--rgw-zone', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--access-key', '8WRXV4QK6AZ20SFM', '--secret', 'q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:17.923 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'create', '--rgw-zone', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--access-key', '8WRXV4QK6AZ20SFM', '--secret', 'q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:17.924 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone create --rgw-zone remove --rgw-zonegroup test-zonegroup --rgw-realm test-realm --access-key 8WRXV4QK6AZ20SFM --secret q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP --cluster c2 --debug-rgw 0 --debug-ms 0 2021-03-11T00:57:18.005 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:57:18.005 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:57:18.060 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:57:18.054+0000 7f5bfd4d2080 0 failed reading obj info from .rgw.root:zone_info.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0: (2) No such file or directory 2021-03-11T00:57:18.060 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:57:18.054+0000 7f5bfd4d2080 0 WARNING: could not read zone params for zone id=4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0 name=test-zone1 2021-03-11T00:57:18.072 INFO:teuthology.orchestra.run.smithi124.stdout:{"id":"7255b2d8-5090-4b6a-9d0f-b07e7cacdfb8","name":"remove","domain_root":"remove.rgw.meta:root","control_pool":"remove.rgw.control","gc_pool":"remove.rgw.log:gc","lc_pool":"remove.rgw.log:lc","log_pool":"remove.rgw.log","intent_log_pool":"remove.rgw.log:intent","usage_log_pool":"remove.rgw.log:usage","roles_pool":"remove.rgw.meta:roles","reshard_pool":"remove.rgw.log:reshard","user_keys_pool":"remove.rgw.meta:users.keys","user_email_pool":"remove.rgw.meta:users.email","user_swift_pool":"remove.rgw.meta:users.swift","user_uid_pool":"remove.rgw.meta:users.uid","otp_pool":"remove.rgw.otp","system_key":{"access_key":"8WRXV4QK6AZ20SFM","secret_key":"q0NmcgkyeCfOFhmQVMhhAzdch7SXYcjP"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"remove.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"remove.rgw.buckets.data"}},"data_extra_pool":"remove.rgw.buckets.non-ec","index_type":0}}],"realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5"} 2021-03-11T00:57:18.073 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:18.082 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:18.083 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone remove --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2021-03-11T00:57:18.155 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:57:18.155 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:57:25.827 INFO:teuthology.orchestra.run.smithi124.stderr:Sending period to new master zone 4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0 2021-03-11T00:57:26.903 INFO:teuthology.orchestra.run.smithi124.stdout:{"id":"76343c7e-8958-4b2a-a007-ca994322980a","epoch":6,"predecessor_uuid":"60afeec2-94d4-4f5a-a646-d47261804cd8","sync_status":["1_1615423843.392949_260.1","","",""],"period_map":{"id":"76343c7e-8958-4b2a-a007-ca994322980a","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","zones":[{"id":"2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05","name":"test-zone3","endpoints":["http://smithi124.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","name":"test-zone1","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"7255b2d8-5090-4b6a-9d0f-b07e7cacdfb8","name":"remove","endpoints":[],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","name":"test-zone2","endpoints":["http://smithi124.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","sync_policy":{"groups":[]}}],"short_zone_ids":[{"key":"2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05","val":906122584},{"key":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","val":3593284871},{"key":"7255b2d8-5090-4b6a-9d0f-b07e7cacdfb8","val":1038570080},{"key":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","val":1094358664}]},"master_zonegroup":"test-zonegroup","master_zone":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","realm_name":"test-realm","realm_epoch":4} 2021-03-11T00:57:26.904 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zonegroup', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--rgw-zone', 'remove', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:26.904 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zonegroup', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--rgw-zone', 'remove', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:26.905 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zonegroup remove --rgw-zonegroup test-zonegroup --rgw-realm test-realm --rgw-zone remove --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T00:57:26.946 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:57:26.946 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:57:26.966 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:57:26.959+0000 7ff77825a080 0 failed reading obj info from .rgw.root:zone_info.2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05: (2) No such file or directory 2021-03-11T00:57:26.966 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:57:26.959+0000 7ff77825a080 0 WARNING: could not read zone params for zone id=2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05 name=test-zone3 2021-03-11T00:57:26.967 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:57:26.959+0000 7ff77825a080 0 failed reading obj info from .rgw.root:zone_info.a2f6d1db-e7f6-4c80-8390-898891b9eb91: (2) No such file or directory 2021-03-11T00:57:26.967 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:57:26.959+0000 7ff77825a080 0 WARNING: could not read zone params for zone id=a2f6d1db-e7f6-4c80-8390-898891b9eb91 name=test-zone2 2021-03-11T00:57:26.974 INFO:teuthology.orchestra.run.smithi119.stdout:{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","zones":[{"id":"2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05","name":"test-zone3","endpoints":["http://smithi124.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","name":"test-zone1","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","name":"test-zone2","endpoints":["http://smithi124.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","sync_policy":{"groups":[]}} 2021-03-11T00:57:26.975 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zonegroup', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--rgw-zone', 'remove', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:26.976 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zonegroup', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--rgw-zone', 'remove', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:26.976 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zonegroup remove --rgw-zonegroup test-zonegroup --rgw-realm test-realm --rgw-zone remove --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T00:57:27.054 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:57:27.054 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:57:27.074 INFO:teuthology.orchestra.run.smithi119.stderr:zone name remove not found in zonegroup test-zonegroup 2021-03-11T00:57:27.082 DEBUG:teuthology.orchestra.run:got remote process result: 2 2021-03-11T00:57:27.083 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'delete', '--rgw-zone', 'remove', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:27.084 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'delete', '--rgw-zone', 'remove', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:27.084 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone delete --rgw-zone remove --cluster c2 --debug-rgw 0 --debug-ms 0 2021-03-11T00:57:27.121 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:57:27.121 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:57:27.168 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:57:27.162+0000 7f06b5ea6080 0 failed reading obj info from .rgw.root:zone_info.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0: (2) No such file or directory 2021-03-11T00:57:27.168 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:57:27.162+0000 7f06b5ea6080 0 WARNING: could not read zone params for zone id=4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0 name=test-zone1 2021-03-11T00:57:27.172 INFO:teuthology.orchestra.run.smithi124.stderr:2021-03-11T00:57:27.166+0000 7f06b5ea6080 0 zone id 7255b2d8-5090-4b6a-9d0f-b07e7cacdfb8 is not a part of zonegroup default 2021-03-11T00:57:27.186 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:27.186 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:27.187 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T00:57:27.228 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:57:27.229 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:57:27.264 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:57:27.259+0000 7f723f55e080 0 WARNING: can't generate connection for zone 7255b2d8-5090-4b6a-9d0f-b07e7cacdfb8 id remove: no endpoints defined 2021-03-11T00:57:27.387 INFO:tasks.rgw.c2.client.0.smithi124.stdout:2021-03-11T00:57:27.378+0000 7fd189fcb700 -1 failed to decode period 2021-03-11T00:57:27.388 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T00:57:27.378+0000 7fcaed75a700 -1 failed to decode period 2021-03-11T00:57:27.407 INFO:teuthology.orchestra.run.smithi119.stdout:{"id":"76343c7e-8958-4b2a-a007-ca994322980a","epoch":7,"predecessor_uuid":"60afeec2-94d4-4f5a-a646-d47261804cd8","sync_status":["1_1615423843.392949_260.1","","",""],"period_map":{"id":"76343c7e-8958-4b2a-a007-ca994322980a","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","zones":[{"id":"2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05","name":"test-zone3","endpoints":["http://smithi124.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","name":"test-zone1","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","name":"test-zone2","endpoints":["http://smithi124.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","sync_policy":{"groups":[]}}],"short_zone_ids":[{"key":"2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05","val":906122584},{"key":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","val":3593284871},{"key":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","val":1094358664}]},"master_zonegroup":"test-zonegroup","master_zone":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","realm_name":"test-realm","realm_epoch":4} 2021-03-11T00:57:27.408 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_zonegroup_remove ... ok 2021-03-11T00:57:27.409 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zonegroup', 'create', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:27.409 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zonegroup', 'create', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:27.410 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zonegroup create --rgw-zonegroup remove_zg --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T00:57:27.486 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:57:27.486 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:57:27.514 INFO:teuthology.orchestra.run.smithi119.stdout:{"id":"4f1cf0e1-dfb0-46d7-bbcd-53d7aac6dfa2","name":"remove_zg","api_name":"remove_zg","is_master":"false","endpoints":[],"hostnames":[],"hostnames_s3website":[],"master_zone":"","zones":[],"placement_targets":[],"default_placement":"","realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","sync_policy":{"groups":[]}} 2021-03-11T00:57:27.516 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'create', '--rgw-zone', 'remove', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:27.517 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'create', '--rgw-zone', 'remove', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:27.517 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone create --rgw-zone remove --rgw-zonegroup remove_zg --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T00:57:27.592 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:57:27.593 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:57:27.652 INFO:teuthology.orchestra.run.smithi119.stdout:{"id":"bc0502cd-38ff-450c-9cb0-1f242fd2c0a7","name":"remove","domain_root":"remove.rgw.meta:root","control_pool":"remove.rgw.control","gc_pool":"remove.rgw.log:gc","lc_pool":"remove.rgw.log:lc","log_pool":"remove.rgw.log","intent_log_pool":"remove.rgw.log:intent","usage_log_pool":"remove.rgw.log:usage","roles_pool":"remove.rgw.meta:roles","reshard_pool":"remove.rgw.log:reshard","user_keys_pool":"remove.rgw.meta:users.keys","user_email_pool":"remove.rgw.meta:users.email","user_swift_pool":"remove.rgw.meta:users.swift","user_uid_pool":"remove.rgw.meta:users.uid","otp_pool":"remove.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"remove.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"remove.rgw.buckets.data"}},"data_extra_pool":"remove.rgw.buckets.non-ec","index_type":0}}],"realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5"} 2021-03-11T00:57:27.652 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:27.653 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:27.654 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T00:57:27.736 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:57:27.736 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:57:27.977 INFO:teuthology.orchestra.run.smithi119.stdout:{"id":"76343c7e-8958-4b2a-a007-ca994322980a","epoch":8,"predecessor_uuid":"60afeec2-94d4-4f5a-a646-d47261804cd8","sync_status":["1_1615423843.392949_260.1","","",""],"period_map":{"id":"76343c7e-8958-4b2a-a007-ca994322980a","zonegroups":[{"id":"4f1cf0e1-dfb0-46d7-bbcd-53d7aac6dfa2","name":"remove_zg","api_name":"remove_zg","is_master":"false","endpoints":[],"hostnames":[],"hostnames_s3website":[],"master_zone":"bc0502cd-38ff-450c-9cb0-1f242fd2c0a7","zones":[{"id":"bc0502cd-38ff-450c-9cb0-1f242fd2c0a7","name":"remove","endpoints":[],"log_meta":"false","log_data":"false","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","sync_policy":{"groups":[]}},{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","zones":[{"id":"2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05","name":"test-zone3","endpoints":["http://smithi124.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","name":"test-zone1","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","name":"test-zone2","endpoints":["http://smithi124.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","sync_policy":{"groups":[]}}],"short_zone_ids":[{"key":"2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05","val":906122584},{"key":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","val":3593284871},{"key":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","val":1094358664},{"key":"bc0502cd-38ff-450c-9cb0-1f242fd2c0a7","val":3861537200}]},"master_zonegroup":"test-zonegroup","master_zone":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","realm_name":"test-realm","realm_epoch":4} 2021-03-11T00:57:27.977 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'delete', '--rgw-zone', 'remove', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:27.978 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'delete', '--rgw-zone', 'remove', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:27.978 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone delete --rgw-zone remove --rgw-zonegroup remove_zg --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T00:57:28.054 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:57:28.055 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:57:28.102 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:57:28.095+0000 7f872a850080 0 zone id bc0502cd-38ff-450c-9cb0-1f242fd2c0a7 is not a part of zonegroup test-zonegroup 2021-03-11T00:57:28.103 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:57:28.095+0000 7f872a850080 0 zone id bc0502cd-38ff-450c-9cb0-1f242fd2c0a7 is not a part of zonegroup default 2021-03-11T00:57:28.115 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:28.116 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:28.117 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T00:57:28.193 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:57:28.193 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:57:28.238 INFO:teuthology.orchestra.run.smithi119.stderr:2021-03-11T00:57:28.231+0000 7f8ff1c9b080 0 ERROR: zonegroup remove_zg has a non existent master zone 2021-03-11T00:57:28.244 DEBUG:teuthology.orchestra.run:got remote process result: 22 2021-03-11T00:57:28.245 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zonegroup', 'delete', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:28.246 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zonegroup', 'delete', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:28.247 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zonegroup delete --rgw-zonegroup remove_zg --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T00:57:28.325 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:57:28.326 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:57:28.357 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:28.358 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:28.359 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T00:57:28.437 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:57:28.437 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:57:28.666 INFO:teuthology.orchestra.run.smithi119.stdout:{"id":"76343c7e-8958-4b2a-a007-ca994322980a","epoch":9,"predecessor_uuid":"60afeec2-94d4-4f5a-a646-d47261804cd8","sync_status":["1_1615423843.392949_260.1","","",""],"period_map":{"id":"76343c7e-8958-4b2a-a007-ca994322980a","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","zones":[{"id":"2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05","name":"test-zone3","endpoints":["http://smithi124.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","name":"test-zone1","endpoints":["http://smithi119.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","name":"test-zone2","endpoints":["http://smithi124.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":11,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","sync_policy":{"groups":[]}}],"short_zone_ids":[{"key":"2cc12c8c-f3ff-4fad-87ea-d3ddecb81b05","val":906122584},{"key":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","val":3593284871},{"key":"a2f6d1db-e7f6-4c80-8390-898891b9eb91","val":1094358664}]},"master_zonegroup":"test-zonegroup","master_zone":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"8cdef83d-c2ff-437f-989e-15e4c59256d5","realm_name":"test-realm","realm_epoch":4} 2021-03-11T00:57:28.667 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_zg_master_zone_delete ... ok 2021-03-11T00:57:28.668 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=nolznk-36 2021-03-11T00:57:37.532 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=nolznk-37 2021-03-11T00:57:37.851 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=nolznk-38 2021-03-11T00:57:37.884 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=nolznk-39 2021-03-11T00:57:43.254 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_set_bucket_website ... rgw_multi.tests.test_set_bucket_policy ... ok 2021-03-11T00:57:43.255 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=nolznk-40 2021-03-11T00:57:43.287 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=nolznk-41 2021-03-11T00:57:43.337 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'disable', '--bucket', 'nolznk-40', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:43.338 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'disable', '--bucket', 'nolznk-40', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:43.338 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync disable --bucket nolznk-40 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T00:57:43.379 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:57:43.379 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:57:43.607 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'disable', '--bucket', 'nolznk-41', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:43.607 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'disable', '--bucket', 'nolznk-41', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:57:43.608 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync disable --bucket nolznk-41 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T00:57:43.690 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:57:43.691 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:57:43.933 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['log', 'list', '--rgw-zone', 'test-zone1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:57:43.933 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'log', 'list', '--rgw-zone', 'test-zone1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:57:43.934 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 log list --rgw-zone test-zone1 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:57:44.010 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:57:44.011 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:57:44.218 INFO:teuthology.orchestra.run.smithi119.stdout:["obj_delete_at_hint.0000000078","meta.history","data.full-sync.index.a2f6d1db-e7f6-4c80-8390-898891b9eb91.1","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.11-nolznk-7","obj_delete_at_hint.0000000070","obj_delete_at_hint.0000000104","obj_delete_at_hint.0000000026","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4930.4-nolznk-39","obj_delete_at_hint.0000000028","obj_delete_at_hint.0000000040","meta.full-sync.index.0","datalog.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91","obj_delete_at_hint.0000000015","obj_delete_at_hint.0000000069","obj_delete_at_hint.0000000095","obj_delete_at_hint.0000000003","obj_delete_at_hint.0000000047","obj_delete_at_hint.0000000035","obj_delete_at_hint.0000000037","obj_delete_at_hint.0000000024","bucket.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91:nolznk-8:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.12:7","bucket.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91:nolznk-33:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4757.3:8","obj_delete_at_hint.0000000000","obj_delete_at_hint.0000000031","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.29-nolznk-25","data_log.3","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.26-nolznk-22","bucket.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91:nolznk-9:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.13:1","bucket.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91:nolznk-8:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.12:3","data.full-sync.index.a2f6d1db-e7f6-4c80-8390-898891b9eb91.2","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.28-nolznk-24","obj_delete_at_hint.0000000076","obj_delete_at_hint.0000000113","obj_delete_at_hint.0000000005","obj_delete_at_hint.0000000011","obj_delete_at_hint.0000000058","obj_delete_at_hint.0000000052","bucket.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91:nolznk-10:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.14:1","bucket.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91:nolznk-7:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.11:1","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4930.6-nolznk-41","obj_delete_at_hint.0000000088","data.full-sync.index.a2f6d1db-e7f6-4c80-8390-898891b9eb91.0","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.30-nolznk-26","obj_delete_at_hint.0000000080","obj_delete_at_hint.0000000090","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.17-nolznk-13","obj_delete_at_hint.0000000110","datalog.sync-status.shard.a2f6d1db-e7f6-4c80-8390-898891b9eb91.3","obj_delete_at_hint.0000000096","bucket.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91:nolznk-16:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.20:8","obj_delete_at_hint.0000000087","obj_delete_at_hint.0000000008","bucket.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91:nolznk-14:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.18:8","obj_delete_at_hint.0000000006","obj_delete_at_hint.0000000029","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4930.2-nolznk-37","obj_delete_at_hint.0000000089","obj_delete_at_hint.0000000022","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.3-nolznk-3","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.14-nolznk-10","bilog.trim","obj_delete_at_hint.0000000016","bucket.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91:nolznk-34:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4757.4:8","obj_delete_at_hint.0000000041","data.full-sync.index.a2f6d1db-e7f6-4c80-8390-898891b9eb91.3","obj_delete_at_hint.0000000018","obj_delete_at_hint.0000000092","obj_delete_at_hint.0000000014","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.31-nolznk-27","obj_delete_at_hint.0000000112","obj_delete_at_hint.0000000007","obj_delete_at_hint.0000000021","obj_delete_at_hint.0000000064","obj_delete_at_hint.0000000071","obj_delete_at_hint.0000000074","obj_delete_at_hint.0000000081","obj_delete_at_hint.0000000009","obj_delete_at_hint.0000000121","data_log.1","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4930.1-nolznk-36","obj_delete_at_hint.0000000125","obj_delete_at_hint.0000000082","meta.full-sync.index.1","obj_delete_at_hint.0000000105","obj_delete_at_hint.0000000059","obj_delete_at_hint.0000000077","bucket.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91:nolznk-26:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.30:10","obj_delete_at_hint.0000000032","2021-03-11-00-a2f6d1db-e7f6-4c80-8390-898891b9eb91.5161.2-nolznk-30","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.12-nolznk-8","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.15-nolznk-11","bucket.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91:nolznk-17:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.21:8","obj_delete_at_hint.0000000053","obj_delete_at_hint.0000000091","bucket.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91:nolznk-18:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.22:8","meta.log.76343c7e-8958-4b2a-a007-ca994322980a.0","obj_delete_at_hint.0000000065","obj_delete_at_hint.0000000083","obj_delete_at_hint.0000000010","mdlog.sync-status.shard.2","bucket.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91:nolznk-12:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.16:1","obj_delete_at_hint.0000000045","obj_delete_at_hint.0000000002","datalog.sync-status.shard.a2f6d1db-e7f6-4c80-8390-898891b9eb91.0.retry","2021-03-11-00-a2f6d1db-e7f6-4c80-8390-898891b9eb91.5161.1-nolznk-29","bucket.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91:nolznk-35:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4839.1:5","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.18-nolznk-14","obj_delete_at_hint.0000000116","obj_delete_at_hint.0000000034","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4757.4-nolznk-34","obj_delete_at_hint.0000000101","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.19-nolznk-15","mdlog.sync-status.shard.1","bucket.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91:nolznk-15:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.19:8","bucket.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91:nolznk-13:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.17:8","obj_delete_at_hint.0000000079","obj_delete_at_hint.0000000049","obj_delete_at_hint.0000000117","obj_delete_at_hint.0000000044","obj_delete_at_hint.0000000066","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4757.3-nolznk-33","bucket.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91:nolznk-11:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.15:5","obj_delete_at_hint.0000000068","obj_delete_at_hint.0000000085","datalog.sync-status.shard.a2f6d1db-e7f6-4c80-8390-898891b9eb91.1","obj_delete_at_hint.0000000073","obj_delete_at_hint.0000000038","meta.full-sync.index.3","obj_delete_at_hint.0000000118","bucket.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91:nolznk-12:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.16:5","obj_delete_at_hint.0000000036","obj_delete_at_hint.0000000103","mdlog.sync-status.shard.3","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.9-nolznk-5","obj_delete_at_hint.0000000119","obj_delete_at_hint.0000000098","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.10-nolznk-6","mdlog.sync-status.shard.0","obj_delete_at_hint.0000000027","datalog.sync-status.shard.a2f6d1db-e7f6-4c80-8390-898891b9eb91.2","mdlog.sync-status","obj_delete_at_hint.0000000019","obj_delete_at_hint.0000000039","bucket.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91:nolznk-25:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.29:10","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4839.1-nolznk-35","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.20-nolznk-16","obj_delete_at_hint.0000000100","obj_delete_at_hint.0000000093","obj_delete_at_hint.0000000004","obj_delete_at_hint.0000000063","obj_delete_at_hint.0000000122","obj_delete_at_hint.0000000057","obj_delete_at_hint.0000000054","obj_delete_at_hint.0000000114","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4757.2-nolznk-32","data_log.0","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.25-nolznk-21","obj_delete_at_hint.0000000012","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.27-nolznk-23","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.24-nolznk-20","bucket.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91:nolznk-12:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.16:3","obj_delete_at_hint.0000000084","obj_delete_at_hint.0000000043","bucket.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91:nolznk-8:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.12:1","obj_delete_at_hint.0000000111","obj_delete_at_hint.0000000048","obj_delete_at_hint.0000000020","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.1-nolznk-1","obj_delete_at_hint.0000000099","obj_delete_at_hint.0000000056","bucket.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91:nolznk-11:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.15:1","datalog.sync-status.shard.a2f6d1db-e7f6-4c80-8390-898891b9eb91.2.retry","obj_delete_at_hint.0000000072","obj_delete_at_hint.0000000062","datalog.sync-status.shard.a2f6d1db-e7f6-4c80-8390-898891b9eb91.0","obj_delete_at_hint.0000000115","obj_delete_at_hint.0000000033","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.16-nolznk-12","obj_delete_at_hint.0000000025","obj_delete_at_hint.0000000123","obj_delete_at_hint.0000000108","obj_delete_at_hint.0000000094","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.23-nolznk-19","obj_delete_at_hint.0000000060","bucket.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91:nolznk-8:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.12:8","obj_delete_at_hint.0000000109","obj_delete_at_hint.0000000013","datalog.sync-status.shard.a2f6d1db-e7f6-4c80-8390-898891b9eb91.3.retry","obj_delete_at_hint.0000000051","obj_delete_at_hint.0000000106","obj_delete_at_hint.0000000017","obj_delete_at_hint.0000000023","obj_delete_at_hint.0000000107","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.2-nolznk-2","obj_delete_at_hint.0000000046","meta.full-sync.index.2","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.22-nolznk-18","bucket.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91:nolznk-7:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.11:8","data_log.2","obj_delete_at_hint.0000000067","datalog.sync-status.shard.a2f6d1db-e7f6-4c80-8390-898891b9eb91.1.retry","bucket.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91:nolznk-7:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.11:7","obj_delete_at_hint.0000000086","bucket.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91:nolznk-13:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.17:5","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4757.1-nolznk-31","bucket.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91:nolznk-7:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.11:3","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.13-nolznk-9","obj_delete_at_hint.0000000050","obj_delete_at_hint.0000000102","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.4-nolznk-4","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.21-nolznk-17","obj_delete_at_hint.0000000124","obj_delete_at_hint.0000000001","obj_delete_at_hint.0000000126","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4930.5-nolznk-40","obj_delete_at_hint.0000000055","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4930.3-nolznk-38","obj_delete_at_hint.0000000061","obj_delete_at_hint.0000000097","obj_delete_at_hint.0000000042","obj_delete_at_hint.0000000075","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.32-nolznk-28","obj_delete_at_hint.0000000120","obj_delete_at_hint.0000000030","bucket.sync-status.a2f6d1db-e7f6-4c80-8390-898891b9eb91:nolznk-11:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.15:3"] 2021-03-11T00:57:44.244 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['log', 'list', '--rgw-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:57:44.244 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'log', 'list', '--rgw-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:57:44.245 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 log list --rgw-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:57:44.281 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:57:44.282 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:57:44.478 INFO:teuthology.orchestra.run.smithi124.stdout:["obj_delete_at_hint.0000000078","meta.history","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.11-nolznk-7","obj_delete_at_hint.0000000070","obj_delete_at_hint.0000000104","obj_delete_at_hint.0000000026","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-11:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.15:3","datalog.sync-status.shard.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.3","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-17:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.21:8","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4930.4-nolznk-39","obj_delete_at_hint.0000000028","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-16:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.20:8","obj_delete_at_hint.0000000040","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-7:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.11:1","meta.full-sync.index.0","obj_delete_at_hint.0000000015","obj_delete_at_hint.0000000069","obj_delete_at_hint.0000000095","obj_delete_at_hint.0000000003","obj_delete_at_hint.0000000047","obj_delete_at_hint.0000000035","obj_delete_at_hint.0000000037","obj_delete_at_hint.0000000024","obj_delete_at_hint.0000000000","obj_delete_at_hint.0000000031","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.29-nolznk-25","data_log.3","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.26-nolznk-22","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.28-nolznk-24","obj_delete_at_hint.0000000076","obj_delete_at_hint.0000000113","obj_delete_at_hint.0000000005","obj_delete_at_hint.0000000011","obj_delete_at_hint.0000000058","obj_delete_at_hint.0000000052","data.full-sync.index.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.3","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-7:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.11:7","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-11:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.15:1","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4930.6-nolznk-41","obj_delete_at_hint.0000000088","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.30-nolznk-26","obj_delete_at_hint.0000000080","obj_delete_at_hint.0000000090","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.17-nolznk-13","obj_delete_at_hint.0000000110","obj_delete_at_hint.0000000096","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-13:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.17:8","obj_delete_at_hint.0000000087","obj_delete_at_hint.0000000008","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-10:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.14:1","datalog.sync-status.shard.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.0","obj_delete_at_hint.0000000006","obj_delete_at_hint.0000000029","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4930.2-nolznk-37","obj_delete_at_hint.0000000089","obj_delete_at_hint.0000000022","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.3-nolznk-3","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.14-nolznk-10","bilog.trim","obj_delete_at_hint.0000000016","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-9:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.13:1","obj_delete_at_hint.0000000041","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-26:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.30:10","obj_delete_at_hint.0000000018","obj_delete_at_hint.0000000092","obj_delete_at_hint.0000000014","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.31-nolznk-27","obj_delete_at_hint.0000000112","obj_delete_at_hint.0000000007","obj_delete_at_hint.0000000021","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-35:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4839.1:5","obj_delete_at_hint.0000000064","obj_delete_at_hint.0000000071","obj_delete_at_hint.0000000074","obj_delete_at_hint.0000000081","obj_delete_at_hint.0000000009","obj_delete_at_hint.0000000121","data_log.1","obj_delete_at_hint.0000000125","obj_delete_at_hint.0000000082","meta.full-sync.index.1","obj_delete_at_hint.0000000105","obj_delete_at_hint.0000000059","obj_delete_at_hint.0000000077","obj_delete_at_hint.0000000032","2021-03-11-00-a2f6d1db-e7f6-4c80-8390-898891b9eb91.5161.2-nolznk-30","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-8:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.12:8","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.12-nolznk-8","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.15-nolznk-11","obj_delete_at_hint.0000000053","obj_delete_at_hint.0000000091","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-14:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.18:8","meta.log.76343c7e-8958-4b2a-a007-ca994322980a.0","obj_delete_at_hint.0000000065","obj_delete_at_hint.0000000083","obj_delete_at_hint.0000000010","datalog.sync-status.shard.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.1.retry","datalog.sync-status.shard.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.2.retry","mdlog.sync-status.shard.2","obj_delete_at_hint.0000000045","obj_delete_at_hint.0000000002","2021-03-11-00-a2f6d1db-e7f6-4c80-8390-898891b9eb91.5161.1-nolznk-29","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.18-nolznk-14","obj_delete_at_hint.0000000116","obj_delete_at_hint.0000000034","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-33:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4757.3:8","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4757.4-nolznk-34","obj_delete_at_hint.0000000101","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-13:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.17:5","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.19-nolznk-15","mdlog.sync-status.shard.1","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-12:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.16:5","obj_delete_at_hint.0000000079","obj_delete_at_hint.0000000049","obj_delete_at_hint.0000000117","obj_delete_at_hint.0000000044","obj_delete_at_hint.0000000066","data.full-sync.index.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.2","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4757.3-nolznk-33","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-8:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.12:3","obj_delete_at_hint.0000000068","obj_delete_at_hint.0000000085","obj_delete_at_hint.0000000073","obj_delete_at_hint.0000000038","meta.full-sync.index.3","obj_delete_at_hint.0000000118","obj_delete_at_hint.0000000036","obj_delete_at_hint.0000000103","mdlog.sync-status.shard.3","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.9-nolznk-5","obj_delete_at_hint.0000000119","obj_delete_at_hint.0000000098","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.10-nolznk-6","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-7:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.11:3","mdlog.sync-status.shard.0","data.full-sync.index.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.0","datalog.sync-status.shard.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.2","obj_delete_at_hint.0000000027","mdlog.sync-status","obj_delete_at_hint.0000000019","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-12:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.16:1","obj_delete_at_hint.0000000039","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4839.1-nolznk-35","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.20-nolznk-16","obj_delete_at_hint.0000000100","datalog.sync-status.shard.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.1","obj_delete_at_hint.0000000093","obj_delete_at_hint.0000000004","obj_delete_at_hint.0000000063","obj_delete_at_hint.0000000122","obj_delete_at_hint.0000000057","obj_delete_at_hint.0000000054","obj_delete_at_hint.0000000114","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4757.2-nolznk-32","data_log.0","obj_delete_at_hint.0000000012","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.24-nolznk-20","obj_delete_at_hint.0000000084","obj_delete_at_hint.0000000043","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-25:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.29:10","obj_delete_at_hint.0000000111","obj_delete_at_hint.0000000048","obj_delete_at_hint.0000000020","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.1-nolznk-1","obj_delete_at_hint.0000000099","obj_delete_at_hint.0000000056","datalog.sync-status.shard.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.3.retry","obj_delete_at_hint.0000000072","obj_delete_at_hint.0000000062","obj_delete_at_hint.0000000115","obj_delete_at_hint.0000000033","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.16-nolznk-12","obj_delete_at_hint.0000000025","obj_delete_at_hint.0000000123","datalog.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","obj_delete_at_hint.0000000108","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-7:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.11:8","obj_delete_at_hint.0000000094","obj_delete_at_hint.0000000060","obj_delete_at_hint.0000000109","obj_delete_at_hint.0000000013","data.full-sync.index.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.1","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-8:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.12:1","obj_delete_at_hint.0000000051","obj_delete_at_hint.0000000106","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-12:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.16:3","obj_delete_at_hint.0000000017","obj_delete_at_hint.0000000023","obj_delete_at_hint.0000000107","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.2-nolznk-2","obj_delete_at_hint.0000000046","meta.full-sync.index.2","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.22-nolznk-18","datalog.sync-status.shard.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.0.retry","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-34:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4757.4:8","data_log.2","obj_delete_at_hint.0000000067","obj_delete_at_hint.0000000086","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-18:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.22:8","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-8:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.12:7","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4757.1-nolznk-31","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.13-nolznk-9","obj_delete_at_hint.0000000050","obj_delete_at_hint.0000000102","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.4-nolznk-4","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.21-nolznk-17","obj_delete_at_hint.0000000124","obj_delete_at_hint.0000000001","obj_delete_at_hint.0000000126","obj_delete_at_hint.0000000055","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-15:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.19:8","obj_delete_at_hint.0000000061","obj_delete_at_hint.0000000097","obj_delete_at_hint.0000000042","obj_delete_at_hint.0000000075","2021-03-11-00-4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.32-nolznk-28","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-11:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.15:5","obj_delete_at_hint.0000000120","obj_delete_at_hint.0000000030"] 2021-03-11T00:57:44.497 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['log', 'list', '--rgw-zone', 'test-zone3', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:57:44.498 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'log', 'list', '--rgw-zone', 'test-zone3', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:57:44.499 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 log list --rgw-zone test-zone3 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:57:44.572 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:57:44.573 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:57:44.773 INFO:teuthology.orchestra.run.smithi124.stdout:["obj_delete_at_hint.0000000078","meta.history","obj_delete_at_hint.0000000070","obj_delete_at_hint.0000000104","obj_delete_at_hint.0000000026","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-11:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.15:3","datalog.sync-status.shard.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.3","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-17:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.21:8","obj_delete_at_hint.0000000028","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-16:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.20:8","obj_delete_at_hint.0000000040","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-7:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.11:1","meta.full-sync.index.0","obj_delete_at_hint.0000000015","obj_delete_at_hint.0000000069","obj_delete_at_hint.0000000095","obj_delete_at_hint.0000000003","obj_delete_at_hint.0000000047","obj_delete_at_hint.0000000035","obj_delete_at_hint.0000000037","obj_delete_at_hint.0000000024","obj_delete_at_hint.0000000000","obj_delete_at_hint.0000000031","obj_delete_at_hint.0000000076","obj_delete_at_hint.0000000113","obj_delete_at_hint.0000000005","obj_delete_at_hint.0000000011","obj_delete_at_hint.0000000058","obj_delete_at_hint.0000000052","data.full-sync.index.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.3","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-7:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.11:7","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-11:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.15:1","obj_delete_at_hint.0000000088","obj_delete_at_hint.0000000080","obj_delete_at_hint.0000000090","obj_delete_at_hint.0000000110","obj_delete_at_hint.0000000096","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-13:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.17:8","obj_delete_at_hint.0000000087","obj_delete_at_hint.0000000008","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-10:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.14:1","datalog.sync-status.shard.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.0","obj_delete_at_hint.0000000006","obj_delete_at_hint.0000000029","obj_delete_at_hint.0000000089","obj_delete_at_hint.0000000022","bilog.trim","obj_delete_at_hint.0000000016","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-9:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.13:1","obj_delete_at_hint.0000000041","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-26:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.30:10","obj_delete_at_hint.0000000018","obj_delete_at_hint.0000000092","obj_delete_at_hint.0000000014","obj_delete_at_hint.0000000112","obj_delete_at_hint.0000000007","obj_delete_at_hint.0000000021","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-35:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4839.1:5","obj_delete_at_hint.0000000064","obj_delete_at_hint.0000000071","obj_delete_at_hint.0000000074","meta.log.470410ef-8c66-4339-a004-064d3e2c6c91.0","obj_delete_at_hint.0000000081","obj_delete_at_hint.0000000009","obj_delete_at_hint.0000000121","obj_delete_at_hint.0000000125","obj_delete_at_hint.0000000082","meta.full-sync.index.1","obj_delete_at_hint.0000000105","obj_delete_at_hint.0000000059","obj_delete_at_hint.0000000077","obj_delete_at_hint.0000000032","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-8:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.12:8","obj_delete_at_hint.0000000053","obj_delete_at_hint.0000000091","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-14:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.18:8","meta.log.76343c7e-8958-4b2a-a007-ca994322980a.0","obj_delete_at_hint.0000000065","obj_delete_at_hint.0000000083","obj_delete_at_hint.0000000010","datalog.sync-status.shard.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.1.retry","datalog.sync-status.shard.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.2.retry","mdlog.sync-status.shard.2","obj_delete_at_hint.0000000045","obj_delete_at_hint.0000000002","obj_delete_at_hint.0000000116","obj_delete_at_hint.0000000034","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-33:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4757.3:8","meta.log.60afeec2-94d4-4f5a-a646-d47261804cd8.0","obj_delete_at_hint.0000000101","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-13:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.17:5","mdlog.sync-status.shard.1","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-12:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.16:5","obj_delete_at_hint.0000000079","obj_delete_at_hint.0000000049","obj_delete_at_hint.0000000117","obj_delete_at_hint.0000000044","obj_delete_at_hint.0000000066","data.full-sync.index.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.2","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-8:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.12:3","obj_delete_at_hint.0000000068","obj_delete_at_hint.0000000085","obj_delete_at_hint.0000000073","obj_delete_at_hint.0000000038","meta.full-sync.index.3","obj_delete_at_hint.0000000118","obj_delete_at_hint.0000000036","obj_delete_at_hint.0000000103","mdlog.sync-status.shard.3","obj_delete_at_hint.0000000119","obj_delete_at_hint.0000000098","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-7:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.11:3","mdlog.sync-status.shard.0","data.full-sync.index.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.0","datalog.sync-status.shard.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.2","obj_delete_at_hint.0000000027","mdlog.sync-status","obj_delete_at_hint.0000000019","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-12:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.16:1","obj_delete_at_hint.0000000039","obj_delete_at_hint.0000000100","datalog.sync-status.shard.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.1","obj_delete_at_hint.0000000093","obj_delete_at_hint.0000000004","obj_delete_at_hint.0000000063","obj_delete_at_hint.0000000122","obj_delete_at_hint.0000000057","obj_delete_at_hint.0000000054","obj_delete_at_hint.0000000114","obj_delete_at_hint.0000000012","obj_delete_at_hint.0000000084","obj_delete_at_hint.0000000043","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-25:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.29:10","obj_delete_at_hint.0000000111","obj_delete_at_hint.0000000048","obj_delete_at_hint.0000000020","obj_delete_at_hint.0000000099","obj_delete_at_hint.0000000056","datalog.sync-status.shard.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.3.retry","obj_delete_at_hint.0000000072","obj_delete_at_hint.0000000062","obj_delete_at_hint.0000000115","obj_delete_at_hint.0000000033","obj_delete_at_hint.0000000025","obj_delete_at_hint.0000000123","datalog.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0","obj_delete_at_hint.0000000108","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-7:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.11:8","obj_delete_at_hint.0000000094","obj_delete_at_hint.0000000060","obj_delete_at_hint.0000000109","obj_delete_at_hint.0000000013","data.full-sync.index.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.1","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-8:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.12:1","obj_delete_at_hint.0000000051","obj_delete_at_hint.0000000106","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-12:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.16:3","obj_delete_at_hint.0000000017","obj_delete_at_hint.0000000023","obj_delete_at_hint.0000000107","obj_delete_at_hint.0000000046","meta.full-sync.index.2","datalog.sync-status.shard.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.0.retry","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-34:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4757.4:8","obj_delete_at_hint.0000000067","obj_delete_at_hint.0000000086","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-18:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.22:8","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-8:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.12:7","obj_delete_at_hint.0000000050","obj_delete_at_hint.0000000102","meta.log.470410ef-8c66-4339-a004-064d3e2c6c91.1","obj_delete_at_hint.0000000124","obj_delete_at_hint.0000000001","obj_delete_at_hint.0000000126","obj_delete_at_hint.0000000055","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-15:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.19:8","obj_delete_at_hint.0000000061","obj_delete_at_hint.0000000097","obj_delete_at_hint.0000000042","obj_delete_at_hint.0000000075","bucket.sync-status.4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-11:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4276.15:5","obj_delete_at_hint.0000000120","obj_delete_at_hint.0000000030"] 2021-03-11T00:57:44.829 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:57:44.830 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:57:44.830 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:57:44.868 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:57:44.868 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:57:45.058 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615424263.897776_164.3","last_update":"2021-03-11T00:57:43.897776Z"},{"marker":"1_1615424263.897776_484.3","last_update":"2021-03-11T00:57:43.897776Z"},{"marker":"1_1615424263.897776_263.2","last_update":"2021-03-11T00:57:43.897776Z"},{"marker":"1_1615424263.897776_327.3","last_update":"2021-03-11T00:57:43.897776Z"}] 2021-03-11T00:57:45.059 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2021-03-11T00:57:45.059 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:57:45.060 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:57:45.060 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:57:45.104 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:57:45.104 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:57:45.305 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17085014794446772242},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424115.698915_116.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:55:15.698915Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423777.100275_264.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:49:37.100275Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_83.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423920.063551_209.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:52:00.063551Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:57:45.306 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone1: shard=0 master=1_1615424263.897776_164.3 target=1_1615424115.698915_116.1, shard=1 master=1_1615424263.897776_484.3 target=1_1615423777.100275_264.1, shard=2 master=1_1615424263.897776_263.2 target=1_1615423685.919484_83.2, shard=3 master=1_1615424263.897776_327.3 target=1_1615423920.063551_209.1 2021-03-11T00:57:50.307 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:57:50.308 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:57:50.309 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:57:50.353 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:57:50.353 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:57:50.551 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17085014794446772242},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424115.698915_116.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:55:15.698915Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423777.100275_264.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:49:37.100275Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_83.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423920.063551_209.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:52:00.063551Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:57:50.552 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone1: shard=0 master=1_1615424263.897776_164.3 target=1_1615424115.698915_116.1, shard=1 master=1_1615424263.897776_484.3 target=1_1615423777.100275_264.1, shard=2 master=1_1615424263.897776_263.2 target=1_1615423685.919484_83.2, shard=3 master=1_1615424263.897776_327.3 target=1_1615423920.063551_209.1 2021-03-11T00:57:55.553 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:57:55.554 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:57:55.554 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:57:55.597 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:57:55.597 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:57:55.792 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17085014794446772242},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424115.698915_116.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:55:15.698915Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423777.100275_264.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:49:37.100275Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_83.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423920.063551_209.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:52:00.063551Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:57:55.792 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone1: shard=0 master=1_1615424263.897776_164.3 target=1_1615424115.698915_116.1, shard=1 master=1_1615424263.897776_484.3 target=1_1615423777.100275_264.1, shard=2 master=1_1615424263.897776_263.2 target=1_1615423685.919484_83.2, shard=3 master=1_1615424263.897776_327.3 target=1_1615423920.063551_209.1 2021-03-11T00:58:00.793 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:58:00.794 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:58:00.796 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:58:00.841 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:58:00.841 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:58:01.042 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17085014794446772242},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424115.698915_116.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:55:15.698915Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423777.100275_264.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:49:37.100275Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_83.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423920.063551_209.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:52:00.063551Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:58:01.043 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone1: shard=0 master=1_1615424263.897776_164.3 target=1_1615424115.698915_116.1, shard=1 master=1_1615424263.897776_484.3 target=1_1615423777.100275_264.1, shard=2 master=1_1615424263.897776_263.2 target=1_1615423685.919484_83.2, shard=3 master=1_1615424263.897776_327.3 target=1_1615423920.063551_209.1 2021-03-11T00:58:06.044 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:58:06.045 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:58:06.046 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:58:06.087 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:58:06.087 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:58:06.293 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17085014794446772242},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424115.698915_116.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:55:15.698915Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423777.100275_264.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:49:37.100275Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_83.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423920.063551_209.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:52:00.063551Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:58:06.293 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone1: shard=0 master=1_1615424263.897776_164.3 target=1_1615424115.698915_116.1, shard=1 master=1_1615424263.897776_484.3 target=1_1615423777.100275_264.1, shard=2 master=1_1615424263.897776_263.2 target=1_1615423685.919484_83.2, shard=3 master=1_1615424263.897776_327.3 target=1_1615423920.063551_209.1 2021-03-11T00:58:11.295 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:58:11.296 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:58:11.297 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:58:11.340 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:58:11.340 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:58:11.544 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17085014794446772242},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424115.698915_116.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:55:15.698915Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423777.100275_264.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:49:37.100275Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_83.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423920.063551_209.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:52:00.063551Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:58:11.545 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone1: shard=0 master=1_1615424263.897776_164.3 target=1_1615424115.698915_116.1, shard=1 master=1_1615424263.897776_484.3 target=1_1615423777.100275_264.1, shard=2 master=1_1615424263.897776_263.2 target=1_1615423685.919484_83.2, shard=3 master=1_1615424263.897776_327.3 target=1_1615423920.063551_209.1 2021-03-11T00:58:16.547 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:58:16.548 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:58:16.548 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:58:16.590 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:58:16.590 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:58:16.787 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17085014794446772242},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424115.698915_116.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:55:15.698915Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423777.100275_264.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:49:37.100275Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_83.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423920.063551_209.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:52:00.063551Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:58:16.788 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone1: shard=0 master=1_1615424263.897776_164.3 target=1_1615424115.698915_116.1, shard=1 master=1_1615424263.897776_484.3 target=1_1615423777.100275_264.1, shard=2 master=1_1615424263.897776_263.2 target=1_1615423685.919484_83.2, shard=3 master=1_1615424263.897776_327.3 target=1_1615423920.063551_209.1 2021-03-11T00:58:21.789 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:58:21.795 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:58:21.795 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:58:21.838 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:58:21.839 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:58:22.034 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17085014794446772242},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424115.698915_116.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:55:15.698915Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423777.100275_264.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:49:37.100275Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_83.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423920.063551_209.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:52:00.063551Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:58:22.035 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone1: shard=0 master=1_1615424263.897776_164.3 target=1_1615424115.698915_116.1, shard=1 master=1_1615424263.897776_484.3 target=1_1615423777.100275_264.1, shard=2 master=1_1615424263.897776_263.2 target=1_1615423685.919484_83.2, shard=3 master=1_1615424263.897776_327.3 target=1_1615423920.063551_209.1 2021-03-11T00:58:27.036 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:58:27.037 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:58:27.037 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:58:27.081 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:58:27.081 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:58:27.268 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17085014794446772242},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424115.698915_116.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:55:15.698915Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423777.100275_264.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:49:37.100275Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_83.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423920.063551_209.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:52:00.063551Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:58:27.269 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone1: shard=0 master=1_1615424263.897776_164.3 target=1_1615424115.698915_116.1, shard=1 master=1_1615424263.897776_484.3 target=1_1615423777.100275_264.1, shard=2 master=1_1615424263.897776_263.2 target=1_1615423685.919484_83.2, shard=3 master=1_1615424263.897776_327.3 target=1_1615423920.063551_209.1 2021-03-11T00:58:32.270 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:58:32.272 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:58:32.272 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:58:32.315 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:58:32.315 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:58:32.507 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17085014794446772242},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424115.698915_116.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:55:15.698915Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423777.100275_264.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:49:37.100275Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_83.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423920.063551_209.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:52:00.063551Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:58:32.508 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone1: shard=0 master=1_1615424263.897776_164.3 target=1_1615424115.698915_116.1, shard=1 master=1_1615424263.897776_484.3 target=1_1615423777.100275_264.1, shard=2 master=1_1615424263.897776_263.2 target=1_1615423685.919484_83.2, shard=3 master=1_1615424263.897776_327.3 target=1_1615423920.063551_209.1 2021-03-11T00:58:37.509 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:58:37.616 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:58:37.617 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:58:37.664 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:58:37.664 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:58:37.884 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17085014794446772242},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424115.698915_116.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:55:15.698915Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423777.100275_264.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:49:37.100275Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_83.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423920.063551_209.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:52:00.063551Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:58:37.885 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone1: shard=0 master=1_1615424263.897776_164.3 target=1_1615424115.698915_116.1, shard=1 master=1_1615424263.897776_484.3 target=1_1615423777.100275_264.1, shard=2 master=1_1615424263.897776_263.2 target=1_1615423685.919484_83.2, shard=3 master=1_1615424263.897776_327.3 target=1_1615423920.063551_209.1 2021-03-11T00:58:42.886 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:58:42.893 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:58:42.894 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:58:42.935 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:58:42.936 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:58:43.133 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17085014794446772242},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424115.698915_116.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:55:15.698915Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423777.100275_264.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:49:37.100275Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_83.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423920.063551_209.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:52:00.063551Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:58:43.134 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone1: shard=0 master=1_1615424263.897776_164.3 target=1_1615424115.698915_116.1, shard=1 master=1_1615424263.897776_484.3 target=1_1615423777.100275_264.1, shard=2 master=1_1615424263.897776_263.2 target=1_1615423685.919484_83.2, shard=3 master=1_1615424263.897776_327.3 target=1_1615423920.063551_209.1 2021-03-11T00:58:48.136 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:58:48.137 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:58:48.138 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:58:48.182 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:58:48.183 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:58:48.379 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17085014794446772242},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424263.897776_164.3","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:57:43.897776Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615424263.897776_484.3","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:57:43.897776Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615424263.897776_263.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:57:43.897776Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615424263.897776_327.3","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:57:43.897776Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:58:48.380 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2021-03-11T00:58:48.380 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:58:48.381 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:58:48.381 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:58:48.423 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:58:48.423 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:58:48.618 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615424263.897776_164.3","last_update":"2021-03-11T00:57:43.897776Z"},{"marker":"1_1615424263.897776_484.3","last_update":"2021-03-11T00:57:43.897776Z"},{"marker":"1_1615424263.897776_263.2","last_update":"2021-03-11T00:57:43.897776Z"},{"marker":"1_1615424263.897776_327.3","last_update":"2021-03-11T00:57:43.897776Z"}] 2021-03-11T00:58:48.619 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone3 source_zone=test-zone1 2021-03-11T00:58:48.620 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:58:48.620 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:58:48.621 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:58:48.665 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:58:48.665 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:58:48.926 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":6834771546787753244},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424115.698915_116.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:55:15.698915Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423777.100275_264.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:49:37.100275Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_83.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423920.063551_209.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:52:00.063551Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:58:48.926 WARNING:rgw_multi.tests:data of zone test-zone3 behind zone test-zone1: shard=0 master=1_1615424263.897776_164.3 target=1_1615424115.698915_116.1, shard=1 master=1_1615424263.897776_484.3 target=1_1615423777.100275_264.1, shard=2 master=1_1615424263.897776_263.2 target=1_1615423685.919484_83.2, shard=3 master=1_1615424263.897776_327.3 target=1_1615423920.063551_209.1 2021-03-11T00:58:53.928 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:58:53.929 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:58:53.930 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:58:53.971 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:58:53.971 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:58:54.224 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":6834771546787753244},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424115.698915_116.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:55:15.698915Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423777.100275_264.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:49:37.100275Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_83.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423920.063551_209.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:52:00.063551Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:58:54.225 WARNING:rgw_multi.tests:data of zone test-zone3 behind zone test-zone1: shard=0 master=1_1615424263.897776_164.3 target=1_1615424115.698915_116.1, shard=1 master=1_1615424263.897776_484.3 target=1_1615423777.100275_264.1, shard=2 master=1_1615424263.897776_263.2 target=1_1615423685.919484_83.2, shard=3 master=1_1615424263.897776_327.3 target=1_1615423920.063551_209.1 2021-03-11T00:58:59.226 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:58:59.227 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:58:59.228 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:58:59.270 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:58:59.271 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:58:59.528 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":6834771546787753244},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424115.698915_116.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:55:15.698915Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423777.100275_264.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:49:37.100275Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_83.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423920.063551_209.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:52:00.063551Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:58:59.529 WARNING:rgw_multi.tests:data of zone test-zone3 behind zone test-zone1: shard=0 master=1_1615424263.897776_164.3 target=1_1615424115.698915_116.1, shard=1 master=1_1615424263.897776_484.3 target=1_1615423777.100275_264.1, shard=2 master=1_1615424263.897776_263.2 target=1_1615423685.919484_83.2, shard=3 master=1_1615424263.897776_327.3 target=1_1615423920.063551_209.1 2021-03-11T00:59:04.530 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:04.530 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:04.531 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:59:04.574 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:59:04.574 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:59:04.829 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":6834771546787753244},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424115.698915_116.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:55:15.698915Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423777.100275_264.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:49:37.100275Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_83.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423920.063551_209.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:52:00.063551Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:59:04.829 WARNING:rgw_multi.tests:data of zone test-zone3 behind zone test-zone1: shard=0 master=1_1615424263.897776_164.3 target=1_1615424115.698915_116.1, shard=1 master=1_1615424263.897776_484.3 target=1_1615423777.100275_264.1, shard=2 master=1_1615424263.897776_263.2 target=1_1615423685.919484_83.2, shard=3 master=1_1615424263.897776_327.3 target=1_1615423920.063551_209.1 2021-03-11T00:59:09.830 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:09.831 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:09.832 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:59:09.876 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:59:09.876 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:59:10.137 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":6834771546787753244},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424115.698915_116.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:55:15.698915Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423777.100275_264.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:49:37.100275Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_83.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423920.063551_209.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:52:00.063551Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:59:10.137 WARNING:rgw_multi.tests:data of zone test-zone3 behind zone test-zone1: shard=0 master=1_1615424263.897776_164.3 target=1_1615424115.698915_116.1, shard=1 master=1_1615424263.897776_484.3 target=1_1615423777.100275_264.1, shard=2 master=1_1615424263.897776_263.2 target=1_1615423685.919484_83.2, shard=3 master=1_1615424263.897776_327.3 target=1_1615423920.063551_209.1 2021-03-11T00:59:15.138 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:15.139 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:15.140 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:59:15.184 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:59:15.184 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:59:15.435 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":6834771546787753244},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424115.698915_116.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:55:15.698915Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423777.100275_264.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:49:37.100275Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_83.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423920.063551_209.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:52:00.063551Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:59:15.435 WARNING:rgw_multi.tests:data of zone test-zone3 behind zone test-zone1: shard=0 master=1_1615424263.897776_164.3 target=1_1615424115.698915_116.1, shard=1 master=1_1615424263.897776_484.3 target=1_1615423777.100275_264.1, shard=2 master=1_1615424263.897776_263.2 target=1_1615423685.919484_83.2, shard=3 master=1_1615424263.897776_327.3 target=1_1615423920.063551_209.1 2021-03-11T00:59:20.437 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:20.438 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:20.438 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:59:20.484 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:59:20.485 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:59:20.731 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":6834771546787753244},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424115.698915_116.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:55:15.698915Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423777.100275_264.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:49:37.100275Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_83.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423920.063551_209.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:52:00.063551Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:59:20.732 WARNING:rgw_multi.tests:data of zone test-zone3 behind zone test-zone1: shard=0 master=1_1615424263.897776_164.3 target=1_1615424115.698915_116.1, shard=1 master=1_1615424263.897776_484.3 target=1_1615423777.100275_264.1, shard=2 master=1_1615424263.897776_263.2 target=1_1615423685.919484_83.2, shard=3 master=1_1615424263.897776_327.3 target=1_1615423920.063551_209.1 2021-03-11T00:59:25.733 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:25.734 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:25.735 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:59:25.783 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:59:25.784 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:59:26.057 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":6834771546787753244},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424115.698915_116.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:55:15.698915Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423777.100275_264.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:49:37.100275Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_83.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423920.063551_209.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:52:00.063551Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:59:26.058 WARNING:rgw_multi.tests:data of zone test-zone3 behind zone test-zone1: shard=0 master=1_1615424263.897776_164.3 target=1_1615424115.698915_116.1, shard=1 master=1_1615424263.897776_484.3 target=1_1615423777.100275_264.1, shard=2 master=1_1615424263.897776_263.2 target=1_1615423685.919484_83.2, shard=3 master=1_1615424263.897776_327.3 target=1_1615423920.063551_209.1 2021-03-11T00:59:31.060 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:31.061 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:31.062 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:59:31.103 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:59:31.103 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:59:31.362 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":6834771546787753244},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424115.698915_116.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:55:15.698915Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423777.100275_264.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:49:37.100275Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_83.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423920.063551_209.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:52:00.063551Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:59:31.363 WARNING:rgw_multi.tests:data of zone test-zone3 behind zone test-zone1: shard=0 master=1_1615424263.897776_164.3 target=1_1615424115.698915_116.1, shard=1 master=1_1615424263.897776_484.3 target=1_1615423777.100275_264.1, shard=2 master=1_1615424263.897776_263.2 target=1_1615423685.919484_83.2, shard=3 master=1_1615424263.897776_327.3 target=1_1615423920.063551_209.1 2021-03-11T00:59:36.364 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:36.365 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:36.366 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:59:36.407 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:59:36.407 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:59:36.664 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":6834771546787753244},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424115.698915_116.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:55:15.698915Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423777.100275_264.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:49:37.100275Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_83.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423920.063551_209.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:52:00.063551Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:59:36.665 WARNING:rgw_multi.tests:data of zone test-zone3 behind zone test-zone1: shard=0 master=1_1615424263.897776_164.3 target=1_1615424115.698915_116.1, shard=1 master=1_1615424263.897776_484.3 target=1_1615423777.100275_264.1, shard=2 master=1_1615424263.897776_263.2 target=1_1615423685.919484_83.2, shard=3 master=1_1615424263.897776_327.3 target=1_1615423920.063551_209.1 2021-03-11T00:59:41.666 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:41.673 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:41.674 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:59:41.715 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:59:41.715 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:59:41.977 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":6834771546787753244},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424115.698915_116.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:55:15.698915Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615423777.100275_264.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:49:37.100275Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615423685.919484_83.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:48:05.919484Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615423920.063551_209.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:52:00.063551Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:59:41.978 WARNING:rgw_multi.tests:data of zone test-zone3 behind zone test-zone1: shard=0 master=1_1615424263.897776_164.3 target=1_1615424115.698915_116.1, shard=1 master=1_1615424263.897776_484.3 target=1_1615423777.100275_264.1, shard=2 master=1_1615424263.897776_263.2 target=1_1615423685.919484_83.2, shard=3 master=1_1615424263.897776_327.3 target=1_1615423920.063551_209.1 2021-03-11T00:59:46.979 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:46.980 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:46.980 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:59:47.023 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:59:47.023 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:59:47.278 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":6834771546787753244},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424263.897776_164.3","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:57:43.897776Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615424263.897776_484.3","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:57:43.897776Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615424263.897776_263.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:57:43.897776Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615424263.897776_327.3","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:57:43.897776Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:59:47.279 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone3 source_zone=test-zone1 2021-03-11T00:59:47.280 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:47.280 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:47.281 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:59:47.360 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:59:47.360 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:59:47.560 INFO:teuthology.orchestra.run.smithi124.stdout:[{"marker":"1_1615424285.094729_134.3","last_update":"2021-03-11T00:58:05.094729Z"},{"marker":"1_1615424285.094729_413.3","last_update":"2021-03-11T00:58:05.094729Z"},{"marker":"1_1615424285.094729_293.2","last_update":"2021-03-11T00:58:05.094729Z"},{"marker":"1_1615424285.094729_328.3","last_update":"2021-03-11T00:58:05.094729Z"}] 2021-03-11T00:59:47.561 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2021-03-11T00:59:47.561 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:47.562 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:47.563 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:59:47.606 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:59:47.607 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:59:47.793 INFO:teuthology.orchestra.run.smithi119.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":3384081757615347012},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424285.094729_134.3","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:58:05.094729Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615424285.094729_413.3","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:58:05.094729Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615424285.094729_293.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:58:05.094729Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615424285.094729_328.3","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:58:05.094729Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T00:59:47.794 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2021-03-11T00:59:47.795 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_sync_disable ... ok 2021-03-11T00:59:47.799 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=nolznk-42 2021-03-11T00:59:47.828 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=nolznk-43 2021-03-11T00:59:48.287 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-42', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:48.288 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-42', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:48.288 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-42 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:59:48.330 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:59:48.331 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:59:48.552 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":"00000000001.35.5"},{"key":1,"val":""},{"key":2,"val":"00000000001.37.5"},{"key":3,"val":""},{"key":4,"val":"00000000001.33.5"},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":"00000000001.39.5"},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:59:48.557 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-42 2021-03-11T00:59:48.559 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-42', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:48.561 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-42', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:48.580 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-42 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:59:48.640 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:59:48.641 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:59:48.855 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:59:48.856 WARNING:rgw_multi.tests:bucket nolznk-42 zone test-zone2 behind zone test-zone1: shard=0 master=00000000001.35.5 target=, shard=2 master=00000000001.37.5 target=, shard=4 master=00000000001.33.5 target=, shard=7 master=00000000001.39.5 target= 2021-03-11T00:59:48.952 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T00:59:48.944+0000 7fcc6cff9700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T00:59:48.952 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T00:59:48.944+0000 7fcc6ffff700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T00:59:48.953 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T00:59:48.944+0000 7fcc777fe700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T00:59:48.953 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T00:59:48.948+0000 7fcc6dffb700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T00:59:48.953 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T00:59:48.948+0000 7fcc6dffb700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T00:59:53.857 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-42', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:53.858 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-42', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:53.859 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-42 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:59:53.904 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:59:53.904 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:59:54.132 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj3","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.35.5"}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj2","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.37.5"}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj1","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.33.5"}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.39.5"}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:59:54.133 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-42 2021-03-11T00:59:54.133 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-42', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:54.134 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-42', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:54.135 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-42 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:59:54.174 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:59:54.175 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:59:54.369 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":"00000000001.35.5"},{"key":1,"val":""},{"key":2,"val":"00000000001.37.5"},{"key":3,"val":""},{"key":4,"val":"00000000001.33.5"},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":"00000000001.39.5"},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:59:54.369 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-42 2021-03-11T00:59:54.370 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-42', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:54.370 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-42', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:54.371 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-42 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:59:54.416 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:59:54.416 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:59:54.673 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.35.5"}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.37.5"}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.33.5"}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.39.5"}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:59:54.674 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-42 2021-03-11T00:59:54.674 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-42', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:54.675 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-42', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:54.676 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-42 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:59:54.756 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:59:54.756 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:59:54.949 INFO:teuthology.orchestra.run.smithi124.stdout:{"markers":[{"key":0,"val":"00000000001.34.5"},{"key":1,"val":""},{"key":2,"val":"00000000001.42.5"},{"key":3,"val":""},{"key":4,"val":"00000000001.35.5"},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":"00000000001.44.5"},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:59:54.950 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-42 2021-03-11T00:59:54.951 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-42', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:54.951 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-42', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:54.952 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-42 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:59:54.997 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:59:54.997 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:59:55.206 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj3","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.34.5"}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj2","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.42.5"}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj1","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.35.5"}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.44.5"}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:59:55.207 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-42 2021-03-11T00:59:55.207 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-42 zones={test-zone2, test-zone1} 2021-03-11T00:59:55.225 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:59:55.232 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2021-03-11T00:59:55.232 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2021-03-11T00:59:55.233 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2021-03-11T00:59:55.233 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2021-03-11T00:59:55.234 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:59:55.240 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2021-03-11T00:59:55.240 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2021-03-11T00:59:55.240 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2021-03-11T00:59:55.241 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2021-03-11T00:59:55.252 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2021-03-11T00:59:55.309 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2021-03-11T00:59:55.418 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2021-03-11T00:59:55.520 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2021-03-11T00:59:55.574 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2021-03-11T00:59:55.686 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2021-03-11T00:59:55.789 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2021-03-11T00:59:55.843 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2021-03-11T00:59:55.955 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2021-03-11T00:59:56.057 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2021-03-11T00:59:56.117 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2021-03-11T00:59:56.228 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2021-03-11T00:59:56.337 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-42 zones={test-zone2, test-zone1} 2021-03-11T00:59:56.338 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-43', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:56.338 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-43', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:56.339 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-43 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:59:56.379 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:59:56.379 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:59:56.573 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":"00000000001.19.5"},{"key":1,"val":""},{"key":2,"val":"00000000001.24.5"},{"key":3,"val":""},{"key":4,"val":"00000000001.30.5"},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":"00000000001.44.5"},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:59:56.573 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-43 2021-03-11T00:59:56.574 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-43', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:56.575 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-43', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:56.576 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-43 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:59:56.621 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:59:56.621 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:59:56.838 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj3","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.19.5"}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj2","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.24.5"}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj1","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.30.5"}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.44.5"}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:59:56.838 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-43 2021-03-11T00:59:56.839 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-43', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:56.839 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-43', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:56.840 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-43 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:59:56.880 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:59:56.881 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:59:57.078 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":"00000000001.19.5"},{"key":1,"val":""},{"key":2,"val":"00000000001.24.5"},{"key":3,"val":""},{"key":4,"val":"00000000001.30.5"},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":"00000000001.44.5"},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:59:57.079 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-43 2021-03-11T00:59:57.079 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-43', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:57.080 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-43', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:57.080 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-43 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:59:57.126 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:59:57.126 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:59:57.379 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.19.5"}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.24.5"}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.30.5"}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.44.5"}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:59:57.379 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-43 2021-03-11T00:59:57.380 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-43', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:57.380 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-43', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:57.381 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-43 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:59:57.460 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:59:57.460 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:59:57.652 INFO:teuthology.orchestra.run.smithi124.stdout:{"markers":[{"key":0,"val":"00000000001.18.5"},{"key":1,"val":""},{"key":2,"val":"00000000001.24.5"},{"key":3,"val":""},{"key":4,"val":"00000000001.31.5"},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":"00000000001.59.5"},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T00:59:57.653 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-43 2021-03-11T00:59:57.653 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-43', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:57.654 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-43', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T00:59:57.654 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-43 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T00:59:57.693 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:59:57.693 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:59:57.885 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj3","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.18.5"}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj2","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.24.5"}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj1","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.31.5"}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.59.5"}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T00:59:57.886 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-43 2021-03-11T00:59:57.887 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-43 zones={test-zone2, test-zone1} 2021-03-11T00:59:57.897 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T00:59:57.902 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2021-03-11T00:59:57.902 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2021-03-11T00:59:57.903 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2021-03-11T00:59:57.903 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2021-03-11T00:59:57.904 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T00:59:57.909 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2021-03-11T00:59:57.909 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2021-03-11T00:59:57.910 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2021-03-11T00:59:57.910 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2021-03-11T00:59:57.920 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2021-03-11T00:59:57.979 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2021-03-11T00:59:58.091 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2021-03-11T00:59:58.196 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2021-03-11T00:59:58.252 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2021-03-11T00:59:58.368 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2021-03-11T00:59:58.471 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2021-03-11T00:59:58.529 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2021-03-11T00:59:58.634 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2021-03-11T00:59:58.744 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2021-03-11T00:59:58.801 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2021-03-11T00:59:58.909 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2021-03-11T00:59:59.013 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-43 zones={test-zone2, test-zone1} 2021-03-11T00:59:59.014 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'disable', '--bucket', 'nolznk-42', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:59:59.014 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'disable', '--bucket', 'nolznk-42', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:59:59.015 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync disable --bucket nolznk-42 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T00:59:59.056 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:59:59.056 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:59:59.293 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'enable', '--bucket', 'nolznk-42', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:59:59.294 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'enable', '--bucket', 'nolznk-42', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:59:59.294 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync enable --bucket nolznk-42 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T00:59:59.376 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:59:59.377 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:59:59.626 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'disable', '--bucket', 'nolznk-43', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:59:59.627 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'disable', '--bucket', 'nolznk-43', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:59:59.627 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync disable --bucket nolznk-43 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T00:59:59.705 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T00:59:59.705 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T00:59:59.949 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'enable', '--bucket', 'nolznk-43', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:59:59.950 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'enable', '--bucket', 'nolznk-43', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T00:59:59.950 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync enable --bucket nolznk-43 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T01:00:00.032 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:00.032 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:00.655 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-42', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:00.655 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-42', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:00.656 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-42 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:00.700 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:00.701 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:00.899 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":"00000000006.51.5"},{"key":1,"val":"00000000004.48.1"},{"key":2,"val":"00000000006.53.5"},{"key":3,"val":"00000000004.51.1"},{"key":4,"val":"00000000006.49.5"},{"key":5,"val":"00000000004.33.1"},{"key":6,"val":"00000000004.47.1"},{"key":7,"val":"00000000006.55.5"},{"key":8,"val":"00000000004.50.1"},{"key":9,"val":"00000000004.38.1"},{"key":10,"val":"00000000004.48.1"}]} 2021-03-11T01:00:00.899 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-42 2021-03-11T01:00:00.900 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-42', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:00.900 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-42', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:00.901 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-42 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:00.945 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:00.946 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:01.171 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj3","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.35.5"}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj2","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.37.5"}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj1","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.33.5"}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.39.5"}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T01:00:01.172 WARNING:rgw_multi.tests:bucket nolznk-42 zone test-zone2 behind zone test-zone1: shard=0 master=00000000006.51.5 target=00000000001.35.5, shard=1 master=00000000004.48.1 target=, shard=2 master=00000000006.53.5 target=00000000001.37.5, shard=3 master=00000000004.51.1 target=, shard=4 master=00000000006.49.5 target=00000000001.33.5, shard=5 master=00000000004.33.1 target=, shard=6 master=00000000004.47.1 target=, shard=7 master=00000000006.55.5 target=00000000001.39.5, shard=8 master=00000000004.50.1 target=, shard=9 master=00000000004.38.1 target=, shard=10 master=00000000004.48.1 target= 2021-03-11T01:00:06.174 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-42', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:06.193 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-42', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:06.195 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-42 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:06.243 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:06.244 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:06.464 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj7","instance":"","ns":""},"count":3},"inc_marker":{"position":"00000000006.51.5"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.48.1"}}},{"key":2,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj6","instance":"","ns":""},"count":3},"inc_marker":{"position":"00000000006.53.5"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.51.1"}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj5","instance":"","ns":""},"count":3},"inc_marker":{"position":"00000000006.49.5"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.33.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.47.1"}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj8","instance":"","ns":""},"count":3},"inc_marker":{"position":"00000000006.55.5"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.50.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.48.1"}}}] 2021-03-11T01:00:06.464 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-42 2021-03-11T01:00:06.465 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-42', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:06.465 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-42', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:06.466 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-42 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:06.508 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:06.509 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:06.701 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":"00000000006.51.5"},{"key":1,"val":"00000000004.48.1"},{"key":2,"val":"00000000006.53.5"},{"key":3,"val":"00000000004.51.1"},{"key":4,"val":"00000000006.49.5"},{"key":5,"val":"00000000004.33.1"},{"key":6,"val":"00000000004.47.1"},{"key":7,"val":"00000000006.55.5"},{"key":8,"val":"00000000004.50.1"},{"key":9,"val":"00000000004.38.1"},{"key":10,"val":"00000000004.48.1"}]} 2021-03-11T01:00:06.701 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-42 2021-03-11T01:00:06.702 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-42', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:06.703 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-42', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:06.703 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-42 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:06.748 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:06.749 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:07.008 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000006.51.5"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.48.1"}}},{"key":2,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000006.53.5"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.51.1"}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000006.49.5"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.33.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.47.1"}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000006.55.5"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.50.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.48.1"}}}] 2021-03-11T01:00:07.008 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-42 2021-03-11T01:00:07.009 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-42', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:07.009 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-42', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:07.010 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-42 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:07.052 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:07.053 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:07.258 INFO:teuthology.orchestra.run.smithi124.stdout:{"markers":[{"key":0,"val":"00000000002.45.5"},{"key":1,"val":""},{"key":2,"val":"00000000002.52.5"},{"key":3,"val":""},{"key":4,"val":"00000000002.43.5"},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":"00000000002.54.5"},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T01:00:07.259 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-42 2021-03-11T01:00:07.261 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-42', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:07.263 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-42', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:07.266 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-42 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:07.313 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:07.313 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:07.519 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj3","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000002.45.5"}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj2","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000002.52.5"}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj1","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000002.43.5"}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000002.54.5"}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T01:00:07.520 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-42 2021-03-11T01:00:07.521 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-42 zones={test-zone2, test-zone1} 2021-03-11T01:00:07.540 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T01:00:07.546 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2021-03-11T01:00:07.546 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2021-03-11T01:00:07.547 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2021-03-11T01:00:07.547 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2021-03-11T01:00:07.548 DEBUG:tasks.rgw_multi.zone_rados:o=obj5 2021-03-11T01:00:07.548 DEBUG:tasks.rgw_multi.zone_rados:o=obj6 2021-03-11T01:00:07.548 DEBUG:tasks.rgw_multi.zone_rados:o=obj7 2021-03-11T01:00:07.549 DEBUG:tasks.rgw_multi.zone_rados:o=obj8 2021-03-11T01:00:07.551 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T01:00:07.558 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2021-03-11T01:00:07.558 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2021-03-11T01:00:07.559 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2021-03-11T01:00:07.559 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2021-03-11T01:00:07.560 DEBUG:tasks.rgw_multi.zone_rados:o=obj5 2021-03-11T01:00:07.560 DEBUG:tasks.rgw_multi.zone_rados:o=obj6 2021-03-11T01:00:07.560 DEBUG:tasks.rgw_multi.zone_rados:o=obj7 2021-03-11T01:00:07.561 DEBUG:tasks.rgw_multi.zone_rados:o=obj8 2021-03-11T01:00:07.574 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2021-03-11T01:00:07.626 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2021-03-11T01:00:07.733 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2021-03-11T01:00:07.837 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2021-03-11T01:00:07.893 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2021-03-11T01:00:08.007 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2021-03-11T01:00:08.112 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2021-03-11T01:00:08.168 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2021-03-11T01:00:08.281 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2021-03-11T01:00:08.391 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2021-03-11T01:00:08.454 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2021-03-11T01:00:08.578 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2021-03-11T01:00:08.722 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj5 2021-03-11T01:00:08.793 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj5 2021-03-11T01:00:08.921 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj5 2021-03-11T01:00:09.033 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj6 2021-03-11T01:00:09.088 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj6 2021-03-11T01:00:09.208 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj6 2021-03-11T01:00:09.322 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj7 2021-03-11T01:00:09.382 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj7 2021-03-11T01:00:09.492 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj7 2021-03-11T01:00:09.601 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj8 2021-03-11T01:00:09.663 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj8 2021-03-11T01:00:09.773 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj8 2021-03-11T01:00:09.878 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-42 zones={test-zone2, test-zone1} 2021-03-11T01:00:09.880 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-43', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:09.882 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-43', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:09.883 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-43 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:09.925 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:09.926 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:10.123 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":"00000000006.25.5"},{"key":1,"val":"00000000004.42.1"},{"key":2,"val":"00000000006.30.5"},{"key":3,"val":"00000000004.46.1"},{"key":4,"val":"00000000006.36.5"},{"key":5,"val":"00000000004.56.1"},{"key":6,"val":"00000000004.52.1"},{"key":7,"val":"00000000006.54.5"},{"key":8,"val":"00000000004.51.1"},{"key":9,"val":"00000000004.55.1"},{"key":10,"val":"00000000004.39.1"}]} 2021-03-11T01:00:10.124 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-43 2021-03-11T01:00:10.124 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-43', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:10.125 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-43', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:10.125 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-43 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:10.169 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:10.170 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:10.383 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj3","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000006.25.5"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.42.1"}}},{"key":2,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj2","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000006.30.5"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.46.1"}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj1","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000006.36.5"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.52.1"}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000006.54.5"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.51.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}}] 2021-03-11T01:00:10.384 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-43 2021-03-11T01:00:10.384 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-43', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:10.385 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-43', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:10.386 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-43 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:10.426 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:10.426 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:10.633 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":"00000000006.25.5"},{"key":1,"val":"00000000004.42.1"},{"key":2,"val":"00000000006.30.5"},{"key":3,"val":"00000000004.46.1"},{"key":4,"val":"00000000006.36.5"},{"key":5,"val":"00000000004.56.1"},{"key":6,"val":"00000000004.52.1"},{"key":7,"val":"00000000006.54.5"},{"key":8,"val":"00000000004.51.1"},{"key":9,"val":"00000000004.55.1"},{"key":10,"val":"00000000004.39.1"}]} 2021-03-11T01:00:10.634 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-43 2021-03-11T01:00:10.635 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-43', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:10.635 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-43', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:10.636 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-43 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:10.678 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:10.679 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:10.940 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000006.25.5"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.42.1"}}},{"key":2,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000006.30.5"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.46.1"}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000006.36.5"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.52.1"}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000006.54.5"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.51.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}}] 2021-03-11T01:00:10.940 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-43 2021-03-11T01:00:10.942 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-43', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:10.942 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-43', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:10.943 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-43 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:11.025 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:11.026 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:11.214 INFO:teuthology.orchestra.run.smithi124.stdout:{"markers":[{"key":0,"val":"00000000002.21.5"},{"key":1,"val":""},{"key":2,"val":"00000000002.27.5"},{"key":3,"val":""},{"key":4,"val":"00000000002.34.5"},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":"00000000002.63.5"},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T01:00:11.214 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-43 2021-03-11T01:00:11.215 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-43', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:11.215 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-43', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:11.216 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-43 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:11.262 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:11.263 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:11.474 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj3","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000002.21.5"}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj2","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000002.27.5"}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj1","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000002.34.5"}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000002.63.5"}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T01:00:11.475 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-43 2021-03-11T01:00:11.476 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-43 zones={test-zone2, test-zone1} 2021-03-11T01:00:11.496 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T01:00:11.502 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2021-03-11T01:00:11.502 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2021-03-11T01:00:11.503 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2021-03-11T01:00:11.504 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2021-03-11T01:00:11.505 DEBUG:tasks.rgw_multi.zone_rados:o=obj5 2021-03-11T01:00:11.505 DEBUG:tasks.rgw_multi.zone_rados:o=obj6 2021-03-11T01:00:11.506 DEBUG:tasks.rgw_multi.zone_rados:o=obj7 2021-03-11T01:00:11.507 DEBUG:tasks.rgw_multi.zone_rados:o=obj8 2021-03-11T01:00:11.507 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T01:00:11.514 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2021-03-11T01:00:11.515 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2021-03-11T01:00:11.515 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2021-03-11T01:00:11.516 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2021-03-11T01:00:11.516 DEBUG:tasks.rgw_multi.zone_rados:o=obj5 2021-03-11T01:00:11.516 DEBUG:tasks.rgw_multi.zone_rados:o=obj6 2021-03-11T01:00:11.517 DEBUG:tasks.rgw_multi.zone_rados:o=obj7 2021-03-11T01:00:11.517 DEBUG:tasks.rgw_multi.zone_rados:o=obj8 2021-03-11T01:00:11.529 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2021-03-11T01:00:11.595 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2021-03-11T01:00:11.723 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2021-03-11T01:00:11.825 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2021-03-11T01:00:11.885 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2021-03-11T01:00:11.994 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2021-03-11T01:00:12.102 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2021-03-11T01:00:12.164 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2021-03-11T01:00:12.290 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2021-03-11T01:00:12.458 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2021-03-11T01:00:12.545 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2021-03-11T01:00:12.693 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2021-03-11T01:00:12.844 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj5 2021-03-11T01:00:12.933 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj5 2021-03-11T01:00:13.066 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj5 2021-03-11T01:00:13.195 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj6 2021-03-11T01:00:13.296 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj6 2021-03-11T01:00:13.426 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj6 2021-03-11T01:00:13.555 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj7 2021-03-11T01:00:13.623 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj7 2021-03-11T01:00:13.764 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj7 2021-03-11T01:00:13.878 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj8 2021-03-11T01:00:13.944 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj8 2021-03-11T01:00:14.066 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj8 2021-03-11T01:00:14.191 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-43 zones={test-zone2, test-zone1} 2021-03-11T01:00:14.193 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:14.196 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:14.198 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:14.236 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:14.237 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:14.435 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615424400.229316_186.3","last_update":"2021-03-11T01:00:00.229316Z"},{"marker":"1_1615424411.386368_531.2","last_update":"2021-03-11T01:00:11.386368Z"},{"marker":"1_1615424411.386368_294.2","last_update":"2021-03-11T01:00:11.386368Z"},{"marker":"1_1615424411.386368_362.4","last_update":"2021-03-11T01:00:11.386368Z"}] 2021-03-11T01:00:14.436 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2021-03-11T01:00:14.437 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:14.438 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:14.439 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:14.480 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:14.481 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:14.671 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17085014794446772242},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424400.229316_186.3","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:00:00.229316Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615424400.229316_526.3","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:00:00.229316Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615424400.229316_293.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:00:00.229316Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615424400.229316_361.3","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:00:00.229316Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T01:00:14.672 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone1: shard=1 master=1_1615424411.386368_531.2 target=1_1615424400.229316_526.3, shard=2 master=1_1615424411.386368_294.2 target=1_1615424400.229316_293.2, shard=3 master=1_1615424411.386368_362.4 target=1_1615424400.229316_361.3 2021-03-11T01:00:19.673 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:19.684 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:19.685 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:19.729 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:19.730 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:19.936 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17085014794446772242},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424400.229316_186.3","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:00:00.229316Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615424400.229316_526.3","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:00:00.229316Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615424400.229316_293.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:00:00.229316Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615424400.229316_361.3","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:00:00.229316Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T01:00:19.937 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone1: shard=1 master=1_1615424411.386368_531.2 target=1_1615424400.229316_526.3, shard=2 master=1_1615424411.386368_294.2 target=1_1615424400.229316_293.2, shard=3 master=1_1615424411.386368_362.4 target=1_1615424400.229316_361.3 2021-03-11T01:00:24.939 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:24.940 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:24.940 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:24.983 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:24.984 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:25.200 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17085014794446772242},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424400.229316_186.3","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:00:00.229316Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615424411.386368_531.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:00:11.386368Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615424411.386368_294.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:00:11.386368Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615424411.386368_362.4","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:00:11.386368Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T01:00:25.200 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2021-03-11T01:00:25.201 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:25.202 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:25.203 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:25.241 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:25.242 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:25.440 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615424400.229316_186.3","last_update":"2021-03-11T01:00:00.229316Z"},{"marker":"1_1615424411.386368_531.2","last_update":"2021-03-11T01:00:11.386368Z"},{"marker":"1_1615424411.386368_294.2","last_update":"2021-03-11T01:00:11.386368Z"},{"marker":"1_1615424411.386368_362.4","last_update":"2021-03-11T01:00:11.386368Z"}] 2021-03-11T01:00:25.441 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone3 source_zone=test-zone1 2021-03-11T01:00:25.442 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:25.442 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:25.443 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:25.484 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:25.485 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:25.748 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":6834771546787753244},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424400.229316_186.3","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:00:00.229316Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615424411.386368_531.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:00:11.386368Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615424411.386368_294.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:00:11.386368Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615424411.386368_362.4","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:00:11.386368Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T01:00:25.749 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone3 source_zone=test-zone1 2021-03-11T01:00:25.750 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:25.751 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:25.752 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:25.831 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:25.832 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:26.039 INFO:teuthology.orchestra.run.smithi124.stdout:[{"marker":"1_1615424285.094729_134.3","last_update":"2021-03-11T00:58:05.094729Z"},{"marker":"1_1615424417.108891_444.2","last_update":"2021-03-11T01:00:17.108891Z"},{"marker":"1_1615424417.108891_311.2","last_update":"2021-03-11T01:00:17.108891Z"},{"marker":"1_1615424417.108891_346.4","last_update":"2021-03-11T01:00:17.108891Z"}] 2021-03-11T01:00:26.040 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2021-03-11T01:00:26.041 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:26.041 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:26.043 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:26.087 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:26.087 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:26.308 INFO:teuthology.orchestra.run.smithi119.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":3384081757615347012},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424285.094729_134.3","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:58:05.094729Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615424417.108891_444.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:00:17.108891Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615424417.108891_311.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:00:17.108891Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615424417.108891_346.4","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:00:17.108891Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T01:00:26.308 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2021-03-11T01:00:26.309 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_sync_enable_right_after_disable ... ok 2021-03-11T01:00:26.311 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=nolznk-44 2021-03-11T01:00:26.340 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=nolznk-45 2021-03-11T01:00:26.783 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:26.784 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:26.785 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:26.824 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:26.825 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:27.002 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=76343c7e-8958-4b2a-a007-ca994322980a 2021-03-11T01:00:27.033 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615424426.366113_218.1","last_update":"2021-03-11T01:00:26.366113Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T01:00:27.034 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2021-03-11T01:00:27.034 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:27.035 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:27.035 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:27.076 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:27.076 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:27.271 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615424400.219805_205.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T01:00:00.219805Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T01:00:27.272 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615424426.366113_218.1 target=1_1615424400.219805_205.1 2021-03-11T01:00:32.274 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:32.275 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:32.275 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:32.325 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:32.326 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:32.516 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615424426.366113_218.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T01:00:26.366113Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T01:00:32.517 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2021-03-11T01:00:32.518 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T01:00:32.518 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:32.519 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:32.520 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:32.606 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:32.606 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:32.861 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615424426.366113_218.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T01:00:26.366113Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T01:00:32.862 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T01:00:32.863 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-44', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:32.863 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-44', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:32.864 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-44 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:32.906 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:32.906 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:33.110 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":"00000000001.25.5"},{"key":1,"val":""},{"key":2,"val":"00000000001.46.5"},{"key":3,"val":""},{"key":4,"val":"00000000001.28.5"},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":"00000000001.49.5"},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T01:00:33.110 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-44 2021-03-11T01:00:33.111 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:33.112 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:33.112 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:33.157 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:33.157 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:33.369 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj3","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.25.5"}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj2","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.46.5"}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj1","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.28.5"}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.49.5"}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T01:00:33.370 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-44 2021-03-11T01:00:33.370 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-44', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:33.371 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-44', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:33.371 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-44 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:33.413 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:33.414 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:33.618 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":"00000000001.25.5"},{"key":1,"val":""},{"key":2,"val":"00000000001.46.5"},{"key":3,"val":""},{"key":4,"val":"00000000001.28.5"},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":"00000000001.49.5"},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T01:00:33.619 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-44 2021-03-11T01:00:33.620 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:33.620 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:33.621 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:33.665 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:33.666 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:33.918 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.25.5"}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.46.5"}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.28.5"}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.49.5"}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T01:00:33.918 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-44 2021-03-11T01:00:33.919 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:33.919 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:33.920 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:33.998 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:33.999 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:34.211 INFO:teuthology.orchestra.run.smithi124.stdout:{"markers":[{"key":0,"val":"00000000001.44.5"},{"key":1,"val":""},{"key":2,"val":"00000000001.41.5"},{"key":3,"val":""},{"key":4,"val":"00000000001.38.5"},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":"00000000001.50.5"},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T01:00:34.211 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-44 2021-03-11T01:00:34.212 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-44', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:34.212 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-44', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:34.213 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-44 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:34.257 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:34.257 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:34.485 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj3","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.44.5"}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj2","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.41.5"}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj1","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.38.5"}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.50.5"}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T01:00:34.486 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-44 2021-03-11T01:00:34.486 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-44 zones={test-zone2, test-zone1} 2021-03-11T01:00:34.504 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T01:00:34.508 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2021-03-11T01:00:34.509 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2021-03-11T01:00:34.509 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2021-03-11T01:00:34.510 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2021-03-11T01:00:34.510 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T01:00:34.516 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2021-03-11T01:00:34.517 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2021-03-11T01:00:34.517 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2021-03-11T01:00:34.518 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2021-03-11T01:00:34.526 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2021-03-11T01:00:34.580 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2021-03-11T01:00:34.696 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2021-03-11T01:00:34.804 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2021-03-11T01:00:34.860 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2021-03-11T01:00:34.969 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2021-03-11T01:00:35.072 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2021-03-11T01:00:35.128 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2021-03-11T01:00:35.242 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2021-03-11T01:00:35.344 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2021-03-11T01:00:35.399 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2021-03-11T01:00:35.514 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2021-03-11T01:00:35.628 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-44 zones={test-zone2, test-zone1} 2021-03-11T01:00:35.629 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-45', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:35.630 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-45', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:35.630 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-45 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:35.683 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:35.683 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:35.881 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":"00000000001.33.5"},{"key":1,"val":""},{"key":2,"val":"00000000001.37.5"},{"key":3,"val":""},{"key":4,"val":"00000000001.32.5"},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":"00000000001.30.5"},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T01:00:35.884 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-45 2021-03-11T01:00:35.886 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-45', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:35.888 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-45', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:35.890 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-45 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:35.942 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:35.948 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:36.155 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj3","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.33.5"}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj2","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.37.5"}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj1","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.32.5"}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.30.5"}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T01:00:36.155 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-45 2021-03-11T01:00:36.156 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-45', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:36.156 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-45', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:36.157 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-45 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:36.202 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:36.203 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:36.411 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":"00000000001.33.5"},{"key":1,"val":""},{"key":2,"val":"00000000001.37.5"},{"key":3,"val":""},{"key":4,"val":"00000000001.32.5"},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":"00000000001.30.5"},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T01:00:36.411 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-45 2021-03-11T01:00:36.412 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-45', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:36.413 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-45', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:36.414 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-45 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:36.463 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:36.463 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:36.722 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.33.5"}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.37.5"}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.32.5"}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.30.5"}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T01:00:36.723 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-45 2021-03-11T01:00:36.723 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-45', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:36.724 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-45', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:36.724 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-45 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:36.813 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:36.814 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:36.999 INFO:teuthology.orchestra.run.smithi124.stdout:{"markers":[{"key":0,"val":"00000000001.28.5"},{"key":1,"val":""},{"key":2,"val":"00000000001.41.5"},{"key":3,"val":""},{"key":4,"val":"00000000001.37.5"},{"key":5,"val":""},{"key":6,"val":""},{"key":7,"val":"00000000001.26.5"},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T01:00:37.001 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-45 2021-03-11T01:00:37.002 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-45', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:37.002 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-45', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:37.003 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-45 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:37.051 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:37.051 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:37.268 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj3","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.28.5"}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj2","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.41.5"}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj1","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.37.5"}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.26.5"}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T01:00:37.269 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-45 2021-03-11T01:00:37.269 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-45 zones={test-zone2, test-zone1} 2021-03-11T01:00:37.282 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T01:00:37.291 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2021-03-11T01:00:37.292 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2021-03-11T01:00:37.293 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2021-03-11T01:00:37.293 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2021-03-11T01:00:37.294 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T01:00:37.304 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2021-03-11T01:00:37.305 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2021-03-11T01:00:37.306 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2021-03-11T01:00:37.307 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2021-03-11T01:00:37.324 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2021-03-11T01:00:37.391 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2021-03-11T01:00:37.506 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2021-03-11T01:00:37.617 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2021-03-11T01:00:37.670 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2021-03-11T01:00:37.783 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2021-03-11T01:00:37.889 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2021-03-11T01:00:37.945 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2021-03-11T01:00:38.058 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2021-03-11T01:00:38.163 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2021-03-11T01:00:38.215 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2021-03-11T01:00:38.322 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2021-03-11T01:00:38.425 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-45 zones={test-zone2, test-zone1} 2021-03-11T01:00:38.426 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'disable', '--bucket', 'nolznk-44', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T01:00:38.427 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'disable', '--bucket', 'nolznk-44', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T01:00:38.428 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync disable --bucket nolznk-44 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T01:00:38.471 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:38.472 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:38.750 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'disable', '--bucket', 'nolznk-45', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T01:00:38.751 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'disable', '--bucket', 'nolznk-45', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T01:00:38.751 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync disable --bucket nolznk-45 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T01:00:38.825 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:38.825 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:39.073 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:39.073 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:39.074 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:39.152 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:39.153 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:39.317 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=76343c7e-8958-4b2a-a007-ca994322980a 2021-03-11T01:00:39.346 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615424439.031829_222.1","last_update":"2021-03-11T01:00:39.031829Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T01:00:39.346 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2021-03-11T01:00:39.347 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:39.348 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:39.348 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:39.391 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:39.392 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:39.579 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615424426.366113_218.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T01:00:26.366113Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T01:00:39.579 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615424439.031829_222.1 target=1_1615424426.366113_218.1 2021-03-11T01:00:44.581 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:44.582 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:44.582 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:44.627 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:44.628 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:44.805 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615424439.031829_222.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T01:00:39.031829Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T01:00:44.805 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2021-03-11T01:00:44.806 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T01:00:44.807 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:44.807 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:44.808 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:44.889 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:44.890 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:45.147 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615424439.031829_222.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T01:00:39.031829Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T01:00:45.147 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T01:00:45.544 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'enable', '--bucket', 'nolznk-44', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T01:00:45.544 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'enable', '--bucket', 'nolznk-44', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T01:00:45.545 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync enable --bucket nolznk-44 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T01:00:45.586 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:45.586 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:45.827 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'enable', '--bucket', 'nolznk-45', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T01:00:45.827 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'enable', '--bucket', 'nolznk-45', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T01:00:45.828 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync enable --bucket nolznk-45 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T01:00:45.906 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:45.907 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:46.161 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-44', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:46.161 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-44', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:46.162 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-44 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:46.249 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:46.249 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:46.422 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":"00000000006.31.1"},{"key":1,"val":"00000000004.39.1"},{"key":2,"val":"00000000006.55.1"},{"key":3,"val":"00000000004.126.1"},{"key":4,"val":"00000000006.37.1"},{"key":5,"val":"00000000004.40.1"},{"key":6,"val":"00000000004.44.1"},{"key":7,"val":"00000000006.55.1"},{"key":8,"val":"00000000004.56.1"},{"key":9,"val":"00000000004.55.1"},{"key":10,"val":"00000000004.38.1"}]} 2021-03-11T01:00:46.422 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-44 2021-03-11T01:00:46.423 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:46.424 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:46.424 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:46.468 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:46.469 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:46.666 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj7","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000006.31.1"}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj5","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000006.37.1"}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T01:00:46.667 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone2 behind zone test-zone1: shard=1 master=00000000004.39.1 target=, shard=2 master=00000000006.55.1 target=, shard=3 master=00000000004.126.1 target=, shard=5 master=00000000004.40.1 target=, shard=6 master=00000000004.44.1 target=, shard=7 master=00000000006.55.1 target=, shard=9 master=00000000004.55.1 target=, shard=10 master=00000000004.38.1 target= 2021-03-11T01:00:51.668 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:51.669 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:51.670 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:51.716 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:51.716 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:51.929 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj7","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000006.31.1"}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj6","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000006.55.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj5","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000006.37.1"}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj8","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000006.55.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:00:51.930 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone2 behind zone test-zone1: shard=1 master=00000000004.39.1 target=, shard=5 master=00000000004.40.1 target=, shard=9 master=00000000004.55.1 target= 2021-03-11T01:00:56.931 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:56.933 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:00:56.934 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:00:56.977 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:00:56.978 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:00:57.187 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj7","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000006.31.1"}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj6","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000006.55.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj5","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000006.37.1"}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj8","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000006.55.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:00:57.187 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone2 behind zone test-zone1: shard=1 master=00000000004.39.1 target=, shard=5 master=00000000004.40.1 target=, shard=9 master=00000000004.55.1 target= 2021-03-11T01:01:02.189 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:01:02.194 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:01:02.195 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:01:02.241 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:01:02.242 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:01:02.443 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj7","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000006.31.1"}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj6","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000006.55.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj5","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000006.37.1"}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj8","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000006.55.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:01:02.444 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone2 behind zone test-zone1: shard=1 master=00000000004.39.1 target=, shard=5 master=00000000004.40.1 target=, shard=9 master=00000000004.55.1 target= 2021-03-11T01:01:07.445 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:01:07.448 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:01:07.449 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:01:07.495 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:01:07.495 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:01:07.694 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj7","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000006.31.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}},{"key":2,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj6","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000006.55.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj5","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000006.37.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj8","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000006.55.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:01:07.694 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-44 2021-03-11T01:01:07.695 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-44', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:01:07.696 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-44', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:01:07.696 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-44 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:01:07.744 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:01:07.744 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:01:07.926 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":"00000000006.31.1"},{"key":1,"val":"00000000004.39.1"},{"key":2,"val":"00000000006.55.1"},{"key":3,"val":"00000000004.126.1"},{"key":4,"val":"00000000006.37.1"},{"key":5,"val":"00000000004.40.1"},{"key":6,"val":"00000000004.44.1"},{"key":7,"val":"00000000006.55.1"},{"key":8,"val":"00000000004.56.1"},{"key":9,"val":"00000000004.55.1"},{"key":10,"val":"00000000004.38.1"}]} 2021-03-11T01:01:07.926 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-44 2021-03-11T01:01:07.927 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:01:07.928 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:01:07.928 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:01:07.973 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:01:07.974 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:01:08.229 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.26.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}},{"key":2,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.47.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.29.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.50.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:01:08.230 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone3 behind zone test-zone1: shard=0 master=00000000006.31.1 target=00000000002.26.1, shard=2 master=00000000006.55.1 target=00000000002.47.1, shard=4 master=00000000006.37.1 target=00000000002.29.1, shard=7 master=00000000006.55.1 target=00000000002.50.1 2021-03-11T01:01:13.232 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:01:13.241 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:01:13.241 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:01:13.287 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:01:13.287 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:01:13.546 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.26.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}},{"key":2,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.47.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.29.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.50.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:01:13.547 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone3 behind zone test-zone1: shard=0 master=00000000006.31.1 target=00000000002.26.1, shard=2 master=00000000006.55.1 target=00000000002.47.1, shard=4 master=00000000006.37.1 target=00000000002.29.1, shard=7 master=00000000006.55.1 target=00000000002.50.1 2021-03-11T01:01:18.550 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:01:18.562 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:01:18.563 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:01:18.608 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:01:18.608 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:01:18.861 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.26.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}},{"key":2,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.47.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.29.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.50.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:01:18.862 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone3 behind zone test-zone1: shard=0 master=00000000006.31.1 target=00000000002.26.1, shard=2 master=00000000006.55.1 target=00000000002.47.1, shard=4 master=00000000006.37.1 target=00000000002.29.1, shard=7 master=00000000006.55.1 target=00000000002.50.1 2021-03-11T01:01:23.863 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:01:23.865 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:01:23.866 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:01:23.910 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:01:23.911 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:01:24.165 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.26.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}},{"key":2,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.47.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.29.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.50.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:01:24.166 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone3 behind zone test-zone1: shard=0 master=00000000006.31.1 target=00000000002.26.1, shard=2 master=00000000006.55.1 target=00000000002.47.1, shard=4 master=00000000006.37.1 target=00000000002.29.1, shard=7 master=00000000006.55.1 target=00000000002.50.1 2021-03-11T01:01:29.167 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:01:29.168 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:01:29.169 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:01:29.214 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:01:29.214 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:01:29.472 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.26.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}},{"key":2,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.47.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.29.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.50.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:01:29.472 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone3 behind zone test-zone1: shard=0 master=00000000006.31.1 target=00000000002.26.1, shard=2 master=00000000006.55.1 target=00000000002.47.1, shard=4 master=00000000006.37.1 target=00000000002.29.1, shard=7 master=00000000006.55.1 target=00000000002.50.1 2021-03-11T01:01:34.474 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:01:34.483 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:01:34.484 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:01:34.530 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:01:34.530 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:01:34.772 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.26.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}},{"key":2,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.47.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.29.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.50.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:01:34.773 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone3 behind zone test-zone1: shard=0 master=00000000006.31.1 target=00000000002.26.1, shard=2 master=00000000006.55.1 target=00000000002.47.1, shard=4 master=00000000006.37.1 target=00000000002.29.1, shard=7 master=00000000006.55.1 target=00000000002.50.1 2021-03-11T01:01:39.775 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:01:39.784 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:01:39.785 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:01:39.829 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:01:39.829 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:01:40.089 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.26.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}},{"key":2,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.47.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.29.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.50.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:01:40.090 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone3 behind zone test-zone1: shard=0 master=00000000006.31.1 target=00000000002.26.1, shard=2 master=00000000006.55.1 target=00000000002.47.1, shard=4 master=00000000006.37.1 target=00000000002.29.1, shard=7 master=00000000006.55.1 target=00000000002.50.1 2021-03-11T01:01:45.091 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:01:45.092 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:01:45.093 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:01:45.137 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:01:45.137 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:01:45.394 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.26.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}},{"key":2,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.47.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.29.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.50.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:01:45.395 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone3 behind zone test-zone1: shard=0 master=00000000006.31.1 target=00000000002.26.1, shard=2 master=00000000006.55.1 target=00000000002.47.1, shard=4 master=00000000006.37.1 target=00000000002.29.1, shard=7 master=00000000006.55.1 target=00000000002.50.1 2021-03-11T01:01:50.397 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:01:50.401 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:01:50.402 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:01:50.445 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:01:50.446 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:01:50.699 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.26.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}},{"key":2,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.47.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.29.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.50.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:01:50.700 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone3 behind zone test-zone1: shard=0 master=00000000006.31.1 target=00000000002.26.1, shard=2 master=00000000006.55.1 target=00000000002.47.1, shard=4 master=00000000006.37.1 target=00000000002.29.1, shard=7 master=00000000006.55.1 target=00000000002.50.1 2021-03-11T01:01:55.701 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:01:55.702 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:01:55.703 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:01:55.757 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:01:55.758 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:01:56.012 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.26.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}},{"key":2,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.47.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.29.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.50.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:01:56.014 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone3 behind zone test-zone1: shard=0 master=00000000006.31.1 target=00000000002.26.1, shard=2 master=00000000006.55.1 target=00000000002.47.1, shard=4 master=00000000006.37.1 target=00000000002.29.1, shard=7 master=00000000006.55.1 target=00000000002.50.1 2021-03-11T01:02:01.016 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:02:01.018 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:02:01.019 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:02:01.067 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:02:01.067 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:02:01.422 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.26.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}},{"key":2,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.47.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.29.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.50.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:02:01.423 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone3 behind zone test-zone1: shard=0 master=00000000006.31.1 target=00000000002.26.1, shard=2 master=00000000006.55.1 target=00000000002.47.1, shard=4 master=00000000006.37.1 target=00000000002.29.1, shard=7 master=00000000006.55.1 target=00000000002.50.1 2021-03-11T01:02:06.425 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:02:06.426 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:02:06.427 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:02:06.470 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:02:06.470 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:02:06.725 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.26.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}},{"key":2,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.47.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.29.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.50.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:02:06.726 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone3 behind zone test-zone1: shard=0 master=00000000006.31.1 target=00000000002.26.1, shard=2 master=00000000006.55.1 target=00000000002.47.1, shard=4 master=00000000006.37.1 target=00000000002.29.1, shard=7 master=00000000006.55.1 target=00000000002.50.1 2021-03-11T01:02:11.727 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:02:11.728 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:02:11.729 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:02:11.774 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:02:11.775 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:02:12.033 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.26.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}},{"key":2,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.47.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.29.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.50.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:02:12.034 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone3 behind zone test-zone1: shard=0 master=00000000006.31.1 target=00000000002.26.1, shard=2 master=00000000006.55.1 target=00000000002.47.1, shard=4 master=00000000006.37.1 target=00000000002.29.1, shard=7 master=00000000006.55.1 target=00000000002.50.1 2021-03-11T01:02:17.035 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:02:17.043 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:02:17.044 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:02:17.100 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:02:17.100 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:02:17.356 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.26.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}},{"key":2,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.47.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.29.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.50.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:02:17.357 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone3 behind zone test-zone1: shard=0 master=00000000006.31.1 target=00000000002.26.1, shard=2 master=00000000006.55.1 target=00000000002.47.1, shard=4 master=00000000006.37.1 target=00000000002.29.1, shard=7 master=00000000006.55.1 target=00000000002.50.1 2021-03-11T01:02:22.358 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:02:22.360 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:02:22.361 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:02:22.407 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:02:22.407 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:02:22.660 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.26.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}},{"key":2,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.47.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.29.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.50.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:02:22.661 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone3 behind zone test-zone1: shard=0 master=00000000006.31.1 target=00000000002.26.1, shard=2 master=00000000006.55.1 target=00000000002.47.1, shard=4 master=00000000006.37.1 target=00000000002.29.1, shard=7 master=00000000006.55.1 target=00000000002.50.1 2021-03-11T01:02:27.662 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:02:27.664 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:02:27.664 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:02:27.711 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:02:27.712 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:02:27.965 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.26.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}},{"key":2,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.47.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.29.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.50.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:02:27.966 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone3 behind zone test-zone1: shard=0 master=00000000006.31.1 target=00000000002.26.1, shard=2 master=00000000006.55.1 target=00000000002.47.1, shard=4 master=00000000006.37.1 target=00000000002.29.1, shard=7 master=00000000006.55.1 target=00000000002.50.1 2021-03-11T01:02:32.968 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:02:32.969 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:02:32.971 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:02:33.014 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:02:33.015 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:02:33.373 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.26.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}},{"key":2,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.47.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.29.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.50.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:02:33.374 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone3 behind zone test-zone1: shard=0 master=00000000006.31.1 target=00000000002.26.1, shard=2 master=00000000006.55.1 target=00000000002.47.1, shard=4 master=00000000006.37.1 target=00000000002.29.1, shard=7 master=00000000006.55.1 target=00000000002.50.1 2021-03-11T01:02:38.376 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:02:38.377 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:02:38.378 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:02:38.423 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:02:38.425 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:02:38.680 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.26.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}},{"key":2,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.47.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.29.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.50.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:02:38.681 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone3 behind zone test-zone1: shard=0 master=00000000006.31.1 target=00000000002.26.1, shard=2 master=00000000006.55.1 target=00000000002.47.1, shard=4 master=00000000006.37.1 target=00000000002.29.1, shard=7 master=00000000006.55.1 target=00000000002.50.1 2021-03-11T01:02:43.682 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:02:43.683 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:02:43.684 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:02:43.729 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:02:43.729 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:02:43.980 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.26.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}},{"key":2,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.47.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.29.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.50.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:02:43.981 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone3 behind zone test-zone1: shard=0 master=00000000006.31.1 target=00000000002.26.1, shard=2 master=00000000006.55.1 target=00000000002.47.1, shard=4 master=00000000006.37.1 target=00000000002.29.1, shard=7 master=00000000006.55.1 target=00000000002.50.1 2021-03-11T01:02:48.982 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:02:48.983 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:02:48.983 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:02:49.028 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:02:49.029 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:02:49.283 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.26.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}},{"key":2,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.47.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.29.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.50.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:02:49.284 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone3 behind zone test-zone1: shard=0 master=00000000006.31.1 target=00000000002.26.1, shard=2 master=00000000006.55.1 target=00000000002.47.1, shard=4 master=00000000006.37.1 target=00000000002.29.1, shard=7 master=00000000006.55.1 target=00000000002.50.1 2021-03-11T01:02:54.286 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:02:54.300 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:02:54.301 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:02:54.350 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:02:54.350 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:02:54.607 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.26.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}},{"key":2,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.47.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.29.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.50.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:02:54.607 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone3 behind zone test-zone1: shard=0 master=00000000006.31.1 target=00000000002.26.1, shard=2 master=00000000006.55.1 target=00000000002.47.1, shard=4 master=00000000006.37.1 target=00000000002.29.1, shard=7 master=00000000006.55.1 target=00000000002.50.1 2021-03-11T01:02:59.609 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:02:59.623 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:02:59.624 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:02:59.670 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:02:59.671 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:02:59.925 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.26.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}},{"key":2,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.47.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.29.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.50.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:02:59.926 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone3 behind zone test-zone1: shard=0 master=00000000006.31.1 target=00000000002.26.1, shard=2 master=00000000006.55.1 target=00000000002.47.1, shard=4 master=00000000006.37.1 target=00000000002.29.1, shard=7 master=00000000006.55.1 target=00000000002.50.1 2021-03-11T01:03:04.928 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:04.936 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:04.937 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:04.981 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:04.981 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:05.242 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000006.31.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}},{"key":2,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.47.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000006.37.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.50.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:03:05.243 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone3 behind zone test-zone1: shard=2 master=00000000006.55.1 target=00000000002.47.1, shard=7 master=00000000006.55.1 target=00000000002.50.1 2021-03-11T01:03:10.245 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:10.255 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:10.257 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:10.302 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:10.302 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:10.549 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000006.31.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}},{"key":2,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.47.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000006.37.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.50.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:03:10.550 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone3 behind zone test-zone1: shard=2 master=00000000006.55.1 target=00000000002.47.1, shard=7 master=00000000006.55.1 target=00000000002.50.1 2021-03-11T01:03:15.552 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:15.559 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:15.561 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:15.608 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:15.608 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:15.864 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000006.31.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}},{"key":2,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.47.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000006.37.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.50.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:03:15.865 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone3 behind zone test-zone1: shard=2 master=00000000006.55.1 target=00000000002.47.1, shard=7 master=00000000006.55.1 target=00000000002.50.1 2021-03-11T01:03:20.866 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:20.874 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:20.875 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:20.924 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:20.925 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:21.179 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000006.31.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}},{"key":2,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.47.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000006.37.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"stopped","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.50.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:03:21.180 WARNING:rgw_multi.tests:bucket nolznk-44 zone test-zone3 behind zone test-zone1: shard=2 master=00000000006.55.1 target=00000000002.47.1, shard=7 master=00000000006.55.1 target=00000000002.50.1 2021-03-11T01:03:26.181 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:26.190 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:26.191 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:26.235 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:26.235 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:26.492 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000006.31.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.39.1"}}},{"key":2,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000006.55.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.126.1"}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000006.37.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.44.1"}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000006.55.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.56.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.55.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.38.1"}}}] 2021-03-11T01:03:26.493 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-44 2021-03-11T01:03:26.494 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:26.495 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:26.496 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-44 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:26.571 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:26.572 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:26.754 INFO:teuthology.orchestra.run.smithi124.stdout:{"markers":[{"key":0,"val":"00000000004.49.1"},{"key":1,"val":"00000000002.39.1"},{"key":2,"val":"00000000004.48.1"},{"key":3,"val":"00000000002.121.1"},{"key":4,"val":"00000000004.45.1"},{"key":5,"val":"00000000002.41.1"},{"key":6,"val":"00000000002.46.1"},{"key":7,"val":"00000000004.55.1"},{"key":8,"val":"00000000002.49.1"},{"key":9,"val":"00000000002.36.1"},{"key":10,"val":"00000000002.46.1"}]} 2021-03-11T01:03:26.755 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-44 2021-03-11T01:03:26.756 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-44', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:26.757 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-44', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:26.758 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-44 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:26.809 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:26.810 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:27.011 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj7","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000004.49.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.39.1"}}},{"key":2,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj6","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000004.48.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.121.1"}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj5","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000004.45.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.41.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.46.1"}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj8","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000004.55.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.49.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.36.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.46.1"}}}] 2021-03-11T01:03:27.013 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-44 2021-03-11T01:03:27.014 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-44 zones={test-zone2, test-zone1} 2021-03-11T01:03:27.044 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T01:03:27.050 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2021-03-11T01:03:27.051 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2021-03-11T01:03:27.051 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2021-03-11T01:03:27.052 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2021-03-11T01:03:27.052 DEBUG:tasks.rgw_multi.zone_rados:o=obj5 2021-03-11T01:03:27.053 DEBUG:tasks.rgw_multi.zone_rados:o=obj6 2021-03-11T01:03:27.053 DEBUG:tasks.rgw_multi.zone_rados:o=obj7 2021-03-11T01:03:27.054 DEBUG:tasks.rgw_multi.zone_rados:o=obj8 2021-03-11T01:03:27.054 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T01:03:27.063 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2021-03-11T01:03:27.064 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2021-03-11T01:03:27.064 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2021-03-11T01:03:27.065 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2021-03-11T01:03:27.066 DEBUG:tasks.rgw_multi.zone_rados:o=obj5 2021-03-11T01:03:27.066 DEBUG:tasks.rgw_multi.zone_rados:o=obj6 2021-03-11T01:03:27.067 DEBUG:tasks.rgw_multi.zone_rados:o=obj7 2021-03-11T01:03:27.067 DEBUG:tasks.rgw_multi.zone_rados:o=obj8 2021-03-11T01:03:27.081 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2021-03-11T01:03:27.137 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2021-03-11T01:03:27.246 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2021-03-11T01:03:27.353 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2021-03-11T01:03:27.413 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2021-03-11T01:03:27.532 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2021-03-11T01:03:27.641 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2021-03-11T01:03:27.699 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2021-03-11T01:03:27.806 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2021-03-11T01:03:27.912 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2021-03-11T01:03:27.967 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2021-03-11T01:03:28.085 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2021-03-11T01:03:28.197 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj5 2021-03-11T01:03:28.256 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj5 2021-03-11T01:03:28.368 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj5 2021-03-11T01:03:28.472 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj6 2021-03-11T01:03:28.528 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj6 2021-03-11T01:03:28.646 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj6 2021-03-11T01:03:28.753 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj7 2021-03-11T01:03:28.812 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj7 2021-03-11T01:03:28.928 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj7 2021-03-11T01:03:29.034 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj8 2021-03-11T01:03:29.099 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj8 2021-03-11T01:03:29.218 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj8 2021-03-11T01:03:29.334 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-44 zones={test-zone2, test-zone1} 2021-03-11T01:03:29.335 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-45', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:29.336 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-45', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:29.337 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-45 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:29.378 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:29.378 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:29.578 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":"00000000006.39.5"},{"key":1,"val":"00000000004.23.1"},{"key":2,"val":"00000000006.47.5"},{"key":3,"val":"00000000004.36.1"},{"key":4,"val":"00000000006.42.5"},{"key":5,"val":"00000000004.18.1"},{"key":6,"val":"00000000004.32.1"},{"key":7,"val":"00000000006.44.5"},{"key":8,"val":"00000000004.40.1"},{"key":9,"val":"00000000004.37.1"},{"key":10,"val":"00000000004.45.1"}]} 2021-03-11T01:03:29.579 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-45 2021-03-11T01:03:29.579 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-45', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:29.580 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-45', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:29.581 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-45 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:29.625 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:29.625 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:29.829 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj3","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000006.39.5"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.23.1"}}},{"key":2,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj2","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000006.47.5"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.36.1"}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj1","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000006.42.5"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.18.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.32.1"}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000006.44.5"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.37.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.45.1"}}}] 2021-03-11T01:03:29.830 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-45 2021-03-11T01:03:29.831 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-45', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:29.832 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-45', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:29.836 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-45 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:29.879 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:29.880 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:30.070 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":"00000000006.39.5"},{"key":1,"val":"00000000004.23.1"},{"key":2,"val":"00000000006.47.5"},{"key":3,"val":"00000000004.36.1"},{"key":4,"val":"00000000006.42.5"},{"key":5,"val":"00000000004.18.1"},{"key":6,"val":"00000000004.32.1"},{"key":7,"val":"00000000006.44.5"},{"key":8,"val":"00000000004.40.1"},{"key":9,"val":"00000000004.37.1"},{"key":10,"val":"00000000004.45.1"}]} 2021-03-11T01:03:30.071 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-45 2021-03-11T01:03:30.071 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-45', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:30.072 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-45', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:30.073 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-45 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:30.123 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:30.123 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:30.376 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000006.39.5"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.23.1"}}},{"key":2,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000006.47.5"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.36.1"}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000006.42.5"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.18.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.32.1"}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000006.44.5"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.40.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.37.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000004.45.1"}}}] 2021-03-11T01:03:30.376 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-45 2021-03-11T01:03:30.377 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-45', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:30.377 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-45', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:30.378 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-45 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:30.461 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:30.461 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:30.635 INFO:teuthology.orchestra.run.smithi124.stdout:{"markers":[{"key":0,"val":"00000000004.34.1"},{"key":1,"val":"00000000002.20.1"},{"key":2,"val":"00000000004.50.1"},{"key":3,"val":"00000000002.42.1"},{"key":4,"val":"00000000004.46.1"},{"key":5,"val":"00000000002.19.1"},{"key":6,"val":"00000000002.37.1"},{"key":7,"val":"00000000004.38.1"},{"key":8,"val":"00000000002.47.1"},{"key":9,"val":"00000000002.45.1"},{"key":10,"val":"00000000002.51.1"}]} 2021-03-11T01:03:30.636 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-45 2021-03-11T01:03:30.637 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-45', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:30.640 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-45', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:30.642 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-45 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:30.690 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:30.691 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:30.894 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj7","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000004.34.1"}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.20.1"}}},{"key":2,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj6","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000004.50.1"}}},{"key":3,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.42.1"}}},{"key":4,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj5","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000004.46.1"}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.19.1"}}},{"key":6,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.37.1"}}},{"key":7,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj8","instance":"","ns":""},"count":2},"inc_marker":{"position":"00000000004.38.1"}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.47.1"}}},{"key":9,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.45.1"}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000002.51.1"}}}] 2021-03-11T01:03:30.895 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-45 2021-03-11T01:03:30.895 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-45 zones={test-zone2, test-zone1} 2021-03-11T01:03:30.904 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T01:03:30.911 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2021-03-11T01:03:30.912 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2021-03-11T01:03:30.912 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2021-03-11T01:03:30.912 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2021-03-11T01:03:30.913 DEBUG:tasks.rgw_multi.zone_rados:o=obj5 2021-03-11T01:03:30.913 DEBUG:tasks.rgw_multi.zone_rados:o=obj6 2021-03-11T01:03:30.914 DEBUG:tasks.rgw_multi.zone_rados:o=obj7 2021-03-11T01:03:30.914 DEBUG:tasks.rgw_multi.zone_rados:o=obj8 2021-03-11T01:03:30.915 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T01:03:30.921 DEBUG:tasks.rgw_multi.zone_rados:o=obj1 2021-03-11T01:03:30.921 DEBUG:tasks.rgw_multi.zone_rados:o=obj2 2021-03-11T01:03:30.922 DEBUG:tasks.rgw_multi.zone_rados:o=obj3 2021-03-11T01:03:30.922 DEBUG:tasks.rgw_multi.zone_rados:o=obj4 2021-03-11T01:03:30.923 DEBUG:tasks.rgw_multi.zone_rados:o=obj5 2021-03-11T01:03:30.923 DEBUG:tasks.rgw_multi.zone_rados:o=obj6 2021-03-11T01:03:30.923 DEBUG:tasks.rgw_multi.zone_rados:o=obj7 2021-03-11T01:03:30.924 DEBUG:tasks.rgw_multi.zone_rados:o=obj8 2021-03-11T01:03:30.935 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2021-03-11T01:03:30.991 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2021-03-11T01:03:31.103 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj1 2021-03-11T01:03:31.208 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2021-03-11T01:03:31.264 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2021-03-11T01:03:31.375 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj2 2021-03-11T01:03:31.485 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2021-03-11T01:03:31.540 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2021-03-11T01:03:31.654 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj3 2021-03-11T01:03:31.752 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2021-03-11T01:03:31.808 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2021-03-11T01:03:31.915 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj4 2021-03-11T01:03:32.016 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj5 2021-03-11T01:03:32.072 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj5 2021-03-11T01:03:32.180 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj5 2021-03-11T01:03:32.284 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj6 2021-03-11T01:03:32.346 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj6 2021-03-11T01:03:32.454 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj6 2021-03-11T01:03:32.557 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj7 2021-03-11T01:03:32.611 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj7 2021-03-11T01:03:32.725 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj7 2021-03-11T01:03:32.833 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj8 2021-03-11T01:03:32.887 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj8 2021-03-11T01:03:32.999 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj8 2021-03-11T01:03:33.105 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-45 zones={test-zone2, test-zone1} 2021-03-11T01:03:33.105 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:33.106 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:33.107 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:33.145 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:33.146 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:33.340 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615424446.127727_262.3","last_update":"2021-03-11T01:00:46.127727Z"},{"marker":"1_1615424455.395780_572.2","last_update":"2021-03-11T01:00:55.395780Z"},{"marker":"1_1615424455.395780_337.2","last_update":"2021-03-11T01:00:55.395780Z"},{"marker":"1_1615424455.395780_422.4","last_update":"2021-03-11T01:00:55.395780Z"}] 2021-03-11T01:03:33.341 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2021-03-11T01:03:33.342 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:33.342 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:33.343 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:33.387 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:33.387 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:33.581 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":17085014794446772242},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424446.127727_262.3","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:00:46.127727Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615424455.395780_572.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:00:55.395780Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615424455.395780_337.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:00:55.395780Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615424455.395780_422.4","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:00:55.395780Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T01:03:33.582 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2021-03-11T01:03:33.583 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:33.583 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:33.584 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:33.624 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:33.624 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:33.806 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615424446.127727_262.3","last_update":"2021-03-11T01:00:46.127727Z"},{"marker":"1_1615424455.395780_572.2","last_update":"2021-03-11T01:00:55.395780Z"},{"marker":"1_1615424455.395780_337.2","last_update":"2021-03-11T01:00:55.395780Z"},{"marker":"1_1615424455.395780_422.4","last_update":"2021-03-11T01:00:55.395780Z"}] 2021-03-11T01:03:33.807 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone3 source_zone=test-zone1 2021-03-11T01:03:33.808 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:33.809 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:33.810 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:33.854 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:33.855 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:34.114 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":6834771546787753244},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424446.127727_262.3","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:00:46.127727Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615424455.395780_572.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:00:55.395780Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615424455.395780_337.2","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:00:55.395780Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615424455.395780_422.4","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:00:55.395780Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T01:03:34.115 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone3 source_zone=test-zone1 2021-03-11T01:03:34.115 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:34.116 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:34.117 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:34.192 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:34.193 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:34.405 INFO:teuthology.orchestra.run.smithi124.stdout:[{"marker":"1_1615424461.119193_213.6","last_update":"2021-03-11T01:01:01.119193Z"},{"marker":"1_1615424461.119193_466.6","last_update":"2021-03-11T01:01:01.119193Z"},{"marker":"1_1615424461.119193_338.4","last_update":"2021-03-11T01:01:01.119193Z"},{"marker":"1_1615424461.119193_392.6","last_update":"2021-03-11T01:01:01.119193Z"}] 2021-03-11T01:03:34.525 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2021-03-11T01:03:34.526 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:34.527 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:34.528 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:34.591 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:34.592 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:34.786 INFO:teuthology.orchestra.run.smithi119.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":3384081757615347012},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"1_1615424461.119193_213.6","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:01:01.119193Z"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1615424461.119193_466.6","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:01:01.119193Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"1_1615424461.119193_338.4","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:01:01.119193Z"}},{"key":3,"val":{"status":"incremental-sync","marker":"1_1615424461.119193_392.6","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T01:01:01.119193Z"}}]},"full_sync":{"total":0,"complete":0}} 2021-03-11T01:03:34.790 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2021-03-11T01:03:34.796 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_sync_disable_enable ... ok 2021-03-11T01:03:34.816 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=nolznk-46 2021-03-11T01:03:34.849 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=nolznk-47 2021-03-11T01:03:37.399 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-46', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:37.400 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-46', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:37.400 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-46 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:37.442 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:37.443 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:37.624 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":"00000000011.60.5"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T01:03:37.624 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-46 2021-03-11T01:03:37.625 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-46', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:37.625 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-46', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:37.626 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-46 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:37.671 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:37.672 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:37.891 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T01:03:37.892 WARNING:rgw_multi.tests:bucket nolznk-46 zone test-zone2 behind zone test-zone1: shard=5 master=00000000011.60.5 target= 2021-03-11T01:03:42.895 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-46', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:42.896 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-46', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:42.897 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-46 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:42.943 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:42.944 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:43.164 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"MULTIPART","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000011.60.5"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T01:03:43.164 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-46 2021-03-11T01:03:43.165 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-46', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:43.166 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-46', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:43.168 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-46 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:43.210 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:43.210 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:43.402 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":"00000000011.60.5"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T01:03:43.403 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-46 2021-03-11T01:03:43.404 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-46', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:43.404 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-46', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:43.405 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-46 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:43.449 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:43.449 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:43.709 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000011.60.5"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T01:03:43.710 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-46 2021-03-11T01:03:43.711 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-46', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:43.712 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-46', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:43.713 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-46 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:43.790 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:43.791 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:43.999 INFO:teuthology.orchestra.run.smithi124.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":""},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":"00000000001.52.5"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T01:03:44.000 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-46 2021-03-11T01:03:44.001 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-46', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:44.002 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-46', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:44.003 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-46 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:44.045 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:44.045 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:44.254 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"MULTIPART","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.52.5"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T01:03:44.254 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-46 2021-03-11T01:03:44.255 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-46 zones={test-zone2, test-zone1} 2021-03-11T01:03:44.264 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T01:03:44.315 DEBUG:tasks.rgw_multi.zone_rados:o=MULTIPART 2021-03-11T01:03:44.316 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T01:03:44.323 DEBUG:tasks.rgw_multi.zone_rados:o=MULTIPART 2021-03-11T01:03:44.376 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=MULTIPART 2021-03-11T01:03:44.593 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=MULTIPART 2021-03-11T01:03:44.887 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=MULTIPART 2021-03-11T01:03:45.177 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-46 zones={test-zone2, test-zone1} 2021-03-11T01:03:45.178 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_multipart_object_sync ... ok 2021-03-11T01:03:45.179 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=nolznk-48 2021-03-11T01:03:45.301 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:45.302 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:45.303 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:45.345 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:45.345 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:45.510 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=76343c7e-8958-4b2a-a007-ca994322980a 2021-03-11T01:03:45.540 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615424625.199593_238.1","last_update":"2021-03-11T01:03:45.199593Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T01:03:45.541 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2021-03-11T01:03:45.541 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:45.542 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:45.543 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:45.590 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:45.590 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:45.793 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615424614.873388_234.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T01:03:34.873388Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T01:03:45.794 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615424625.199593_238.1 target=1_1615424614.873388_234.1 2021-03-11T01:03:46.316 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T01:03:46.311+0000 7fcc75ffb700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T01:03:46.317 INFO:tasks.rgw.c2.client.0.smithi124.stdout:2021-03-11T01:03:46.311+0000 7fd2b77fe700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T01:03:46.318 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T01:03:46.311+0000 7fcc6f7fe700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T01:03:46.318 INFO:tasks.rgw.c2.client.0.smithi124.stdout:2021-03-11T01:03:46.311+0000 7fd2b1ff3700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T01:03:46.319 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T01:03:46.311+0000 7fcc6e7fc700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T01:03:46.319 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T01:03:46.311+0000 7fcc6b7f6700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T01:03:46.321 INFO:tasks.rgw.c2.client.0.smithi124.stdout:2021-03-11T01:03:46.311+0000 7fd2b5ffb700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T01:03:46.321 INFO:tasks.rgw.c2.client.0.smithi124.stdout:2021-03-11T01:03:46.311+0000 7fd2cdffb700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T01:03:50.795 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:50.796 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:50.797 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:50.839 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:50.839 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:51.046 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615424625.199593_238.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T01:03:45.199593Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T01:03:51.047 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2021-03-11T01:03:51.048 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T01:03:51.049 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:51.050 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:51.051 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:51.126 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:51.126 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:51.379 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615424625.199593_238.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T01:03:45.199593Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T01:03:51.380 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T01:03:51.381 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-48', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:51.381 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-48', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:51.383 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-48 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:51.429 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:51.430 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:51.626 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":"00000000001.23.5"},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":"00000000001.40.5"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":""},{"key":9,"val":""},{"key":10,"val":""}]} 2021-03-11T01:03:51.628 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-48 2021-03-11T01:03:51.629 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-48', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:51.630 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-48', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:51.647 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-48 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:51.691 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:51.691 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:51.910 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"testobj-sse-c","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.23.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"testobj-sse-kms","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.40.5"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2021-03-11T01:03:51.911 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-48 2021-03-11T01:03:52.010 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_encrypted_object_sync ... ok 2021-03-11T01:03:52.011 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=nolznk-49 2021-03-11T01:03:52.240 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:52.241 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:52.242 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:52.284 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:52.284 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:52.450 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=76343c7e-8958-4b2a-a007-ca994322980a 2021-03-11T01:03:52.481 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615424632.031791_242.1","last_update":"2021-03-11T01:03:52.031791Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T01:03:52.482 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2021-03-11T01:03:52.483 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:52.484 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:52.485 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:52.523 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:52.524 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:52.711 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615424632.031791_242.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T01:03:52.031791Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T01:03:52.711 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2021-03-11T01:03:52.712 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T01:03:52.713 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:52.715 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:52.717 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:52.800 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:52.800 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:53.058 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615424625.199593_238.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T01:03:45.199593Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T01:03:53.059 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1615424632.031791_242.1 target=1_1615424625.199593_238.1 2021-03-11T01:03:53.123 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T01:03:53.115+0000 7fcc6c7f8700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T01:03:53.124 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T01:03:53.115+0000 7fcc67fef700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T01:03:53.125 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T01:03:53.115+0000 7fcc68ff1700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T01:03:53.127 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T01:03:53.119+0000 7fcc757fa700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T01:03:53.127 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T01:03:53.119+0000 7fcc80ff9700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T01:03:58.061 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:58.062 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:58.063 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:58.106 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:58.106 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:58.363 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615424632.031791_242.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T01:03:52.031791Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T01:03:58.364 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T01:03:58.365 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-49', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:58.365 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-49', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:58.366 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-49 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:58.406 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:58.605 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":"00000000001.29.5"},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":"00000000001.39.5"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000001.63.5"},{"key":9,"val":""},{"key":10,"val":"00000000001.41.5"}]} 2021-03-11T01:03:58.605 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-49 2021-03-11T01:03:58.606 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-49', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:58.607 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-49', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:58.608 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-49 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:58.660 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:58.660 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:58.901 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"a","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.29.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"d","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.39.5"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"c","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.63.5"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"b","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.41.5"}}}] 2021-03-11T01:03:58.902 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-49 2021-03-11T01:03:58.924 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-49', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:58.926 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-49', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:58.927 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-49 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:58.971 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:58.971 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:59.168 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":"00000000001.29.5"},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":"00000000001.39.5"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000001.63.5"},{"key":9,"val":""},{"key":10,"val":"00000000001.41.5"}]} 2021-03-11T01:03:59.169 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-49 2021-03-11T01:03:59.169 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-49', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:59.170 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-49', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:59.170 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-49 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:59.223 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:59.224 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:59.478 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.29.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.39.5"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.63.5"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.41.5"}}}] 2021-03-11T01:03:59.480 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-49 2021-03-11T01:03:59.483 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-49', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:59.483 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-49', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:59.484 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-49 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:59.563 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:59.564 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:03:59.764 INFO:teuthology.orchestra.run.smithi124.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":"00000000001.35.5"},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":"00000000001.51.5"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000001.55.5"},{"key":9,"val":""},{"key":10,"val":"00000000001.51.5"}]} 2021-03-11T01:03:59.765 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-49 2021-03-11T01:03:59.766 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-49', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:59.767 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-49', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:03:59.770 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-49 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:03:59.817 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:03:59.817 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:04:00.023 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"a","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.35.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"d","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.51.5"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"c","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.55.5"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"b","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.51.5"}}}] 2021-03-11T01:04:00.024 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-49 2021-03-11T01:04:00.024 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-49 zones={test-zone2, test-zone1} 2021-03-11T01:04:00.048 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T01:04:00.055 DEBUG:tasks.rgw_multi.zone_rados:o=a 2021-03-11T01:04:00.056 DEBUG:tasks.rgw_multi.zone_rados:o=b 2021-03-11T01:04:00.056 DEBUG:tasks.rgw_multi.zone_rados:o=c 2021-03-11T01:04:00.057 DEBUG:tasks.rgw_multi.zone_rados:o=d 2021-03-11T01:04:00.057 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T01:04:00.063 DEBUG:tasks.rgw_multi.zone_rados:o=a 2021-03-11T01:04:00.064 DEBUG:tasks.rgw_multi.zone_rados:o=b 2021-03-11T01:04:00.064 DEBUG:tasks.rgw_multi.zone_rados:o=c 2021-03-11T01:04:00.064 DEBUG:tasks.rgw_multi.zone_rados:o=d 2021-03-11T01:04:00.076 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2021-03-11T01:04:00.141 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2021-03-11T01:04:00.253 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2021-03-11T01:04:00.356 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2021-03-11T01:04:00.412 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2021-03-11T01:04:00.528 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2021-03-11T01:04:00.642 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2021-03-11T01:04:00.719 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2021-03-11T01:04:00.841 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2021-03-11T01:04:00.947 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2021-03-11T01:04:01.007 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2021-03-11T01:04:01.122 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2021-03-11T01:04:01.231 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-49 zones={test-zone2, test-zone1} 2021-03-11T01:04:01.231 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'autotrim', '--rgw-sync-log-trim-max-buckets', '0', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T01:04:01.232 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'autotrim', '--rgw-sync-log-trim-max-buckets', '0', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T01:04:01.233 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog autotrim --rgw-sync-log-trim-max-buckets 0 --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T01:04:01.272 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:04:01.273 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:04:01.465 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=nolznk-50 2021-03-11T01:04:01.691 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:01.692 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:01.693 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:04:01.735 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:04:01.735 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:04:01.898 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=76343c7e-8958-4b2a-a007-ca994322980a 2021-03-11T01:04:01.927 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615424641.485395_246.1","last_update":"2021-03-11T01:04:01.485395Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T01:04:01.928 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2021-03-11T01:04:01.929 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:01.930 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:01.931 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:04:01.974 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:04:01.975 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:04:02.186 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615424632.031791_242.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T01:03:52.031791Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T01:04:02.186 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1615424641.485395_246.1 target=1_1615424632.031791_242.1 2021-03-11T01:04:02.531 INFO:tasks.rgw.c2.client.0.smithi124.stdout:2021-03-11T01:04:02.523+0000 7fd2b4ff9700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T01:04:02.536 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T01:04:02.523+0000 7fcc767fc700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T01:04:02.541 INFO:tasks.rgw.c2.client.0.smithi124.stdout:2021-03-11T01:04:02.527+0000 7fd2b67fc700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T01:04:02.542 INFO:tasks.rgw.c2.client.0.smithi124.stdout:2021-03-11T01:04:02.527+0000 7fd2ccff9700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T01:04:02.547 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T01:04:02.527+0000 7fcc82ffd700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T01:04:02.548 INFO:tasks.rgw.c2.client.1.smithi124.stdout:2021-03-11T01:04:02.527+0000 7fcc81ffb700 -1 ERROR: _send_request(): get_sync_policy_handler() returned -2 2021-03-11T01:04:07.188 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:07.196 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:07.197 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:04:07.247 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:04:07.247 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:04:07.452 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615424641.485395_246.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T01:04:01.485395Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T01:04:07.453 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2021-03-11T01:04:07.454 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T01:04:07.458 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:07.459 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:07.461 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:04:07.541 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:04:07.541 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:04:07.793 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615424641.485395_246.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T01:04:01.485395Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T01:04:07.793 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T01:04:07.794 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-50', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:07.795 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-50', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:07.795 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-50 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:04:07.837 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:04:07.838 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:04:08.035 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":"00000000001.42.5"},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":"00000000001.44.5"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000001.32.5"},{"key":9,"val":""},{"key":10,"val":"00000000001.38.5"}]} 2021-03-11T01:04:08.035 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-50 2021-03-11T01:04:08.036 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-50', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:08.036 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-50', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:08.037 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-50 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:04:08.081 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:04:08.081 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:04:08.296 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"a","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.42.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"d","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.44.5"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"c","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.32.5"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"b","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.38.5"}}}] 2021-03-11T01:04:08.297 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=nolznk-50 2021-03-11T01:04:08.298 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-50', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:08.299 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-50', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:08.300 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-50 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:04:08.343 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:04:08.343 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:04:08.545 INFO:teuthology.orchestra.run.smithi119.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":"00000000001.42.5"},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":"00000000001.44.5"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000001.32.5"},{"key":9,"val":""},{"key":10,"val":"00000000001.38.5"}]} 2021-03-11T01:04:08.546 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-50 2021-03-11T01:04:08.547 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-50', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:08.548 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'nolznk-50', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:08.549 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket nolznk-50 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:04:08.590 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:04:08.590 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:04:08.840 INFO:teuthology.orchestra.run.smithi124.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.42.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.44.5"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.32.5"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000001.38.5"}}}] 2021-03-11T01:04:08.841 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=nolznk-50 2021-03-11T01:04:08.841 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-50', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:08.842 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'nolznk-50', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:08.843 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket nolznk-50 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:04:08.924 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:04:08.925 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:04:09.124 INFO:teuthology.orchestra.run.smithi124.stdout:{"markers":[{"key":0,"val":""},{"key":1,"val":"00000000001.56.5"},{"key":2,"val":""},{"key":3,"val":""},{"key":4,"val":""},{"key":5,"val":"00000000001.57.5"},{"key":6,"val":""},{"key":7,"val":""},{"key":8,"val":"00000000001.41.5"},{"key":9,"val":""},{"key":10,"val":"00000000001.52.5"}]} 2021-03-11T01:04:09.125 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-50 2021-03-11T01:04:09.126 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-50', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:09.127 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'nolznk-50', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:09.128 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket nolznk-50 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:04:09.173 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:04:09.174 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:04:09.385 INFO:teuthology.orchestra.run.smithi119.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":1,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"a","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.56.5"}}},{"key":2,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":3,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":4,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":5,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"d","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.57.5"}}},{"key":6,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":7,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":8,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"c","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.41.5"}}},{"key":9,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}},{"key":10,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"b","instance":"","ns":""},"count":1},"inc_marker":{"position":"00000000001.52.5"}}}] 2021-03-11T01:04:09.386 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=nolznk-50 2021-03-11T01:04:09.386 INFO:tasks.rgw_multi.zone_rados:comparing bucket=nolznk-50 zones={test-zone2, test-zone1} 2021-03-11T01:04:09.414 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2021-03-11T01:04:09.421 DEBUG:tasks.rgw_multi.zone_rados:o=a 2021-03-11T01:04:09.422 DEBUG:tasks.rgw_multi.zone_rados:o=b 2021-03-11T01:04:09.423 DEBUG:tasks.rgw_multi.zone_rados:o=c 2021-03-11T01:04:09.423 DEBUG:tasks.rgw_multi.zone_rados:o=d 2021-03-11T01:04:09.424 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2021-03-11T01:04:09.471 DEBUG:tasks.rgw_multi.zone_rados:o=a 2021-03-11T01:04:09.473 DEBUG:tasks.rgw_multi.zone_rados:o=b 2021-03-11T01:04:09.474 DEBUG:tasks.rgw_multi.zone_rados:o=c 2021-03-11T01:04:09.474 DEBUG:tasks.rgw_multi.zone_rados:o=d 2021-03-11T01:04:09.538 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2021-03-11T01:04:09.592 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2021-03-11T01:04:09.706 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2021-03-11T01:04:09.813 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2021-03-11T01:04:09.879 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2021-03-11T01:04:09.990 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2021-03-11T01:04:10.094 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2021-03-11T01:04:10.160 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2021-03-11T01:04:10.270 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2021-03-11T01:04:10.376 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2021-03-11T01:04:10.432 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2021-03-11T01:04:10.541 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2021-03-11T01:04:10.644 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=nolznk-50 zones={test-zone2, test-zone1} 2021-03-11T01:04:10.645 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'autotrim', '--rgw-sync-log-trim-max-buckets', '1', '--rgw-sync-log-trim-min-cold-buckets', '0', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T01:04:10.645 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'autotrim', '--rgw-sync-log-trim-max-buckets', '1', '--rgw-sync-log-trim-min-cold-buckets', '0', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T01:04:10.646 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog autotrim --rgw-sync-log-trim-max-buckets 1 --rgw-sync-log-trim-min-cold-buckets 0 --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T01:04:10.682 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:04:10.683 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:04:10.895 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'list', '--bucket', 'nolznk-50', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:10.896 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'list', '--bucket', 'nolznk-50', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:10.896 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog list --bucket nolznk-50 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:04:10.973 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:04:10.974 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:04:11.165 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2021-03-11T01:04:11.166 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'list', '--bucket', 'nolznk-49', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:11.167 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'list', '--bucket', 'nolznk-49', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:11.168 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog list --bucket nolznk-49 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:04:11.247 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:04:11.247 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:04:11.442 INFO:teuthology.orchestra.run.smithi119.stdout:[{"op_id":"1#00000000001.29.5","op_tag":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4928.1983","op":"write","object":"a","instance":"","state":"complete","index_ver":1,"timestamp":"2021-03-11T01:03:52.039591423Z","ver":{"pool":9,"epoch":4},"bilog_flags":0,"versioned":false,"owner":"","owner_display_name":"","zones_trace":[{"entry":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-49:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4930.14"}]},{"op_id":"5#00000000001.39.5","op_tag":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4928.1989","op":"write","object":"d","instance":"","state":"complete","index_ver":1,"timestamp":"2021-03-11T01:03:52.192191996Z","ver":{"pool":9,"epoch":2},"bilog_flags":0,"versioned":false,"owner":"","owner_display_name":"","zones_trace":[{"entry":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-49:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4930.14"}]},{"op_id":"8#00000000001.63.5","op_tag":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4928.1987","op":"write","object":"c","instance":"","state":"complete","index_ver":1,"timestamp":"2021-03-11T01:03:52.142111120Z","ver":{"pool":9,"epoch":3},"bilog_flags":0,"versioned":false,"owner":"","owner_display_name":"","zones_trace":[{"entry":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-49:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4930.14"}]},{"op_id":"10#00000000001.41.5","op_tag":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4928.1985","op":"write","object":"b","instance":"","state":"complete","index_ver":1,"timestamp":"2021-03-11T01:03:52.089400484Z","ver":{"pool":9,"epoch":1},"bilog_flags":0,"versioned":false,"owner":"","owner_display_name":"","zones_trace":[{"entry":"4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0:nolznk-49:4796a6da-ec9c-43e4-b480-3a8ee0c0d8a0.4930.14"}]}] 2021-03-11T01:04:11.443 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'autotrim', '--rgw-sync-log-trim-max-buckets', '999', '--rgw-sync-log-trim-min-cold-buckets', '999', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T01:04:11.444 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'autotrim', '--rgw-sync-log-trim-max-buckets', '999', '--rgw-sync-log-trim-min-cold-buckets', '999', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2021-03-11T01:04:11.445 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog autotrim --rgw-sync-log-trim-max-buckets 999 --rgw-sync-log-trim-min-cold-buckets 999 --cluster c1 --debug-rgw 0 --debug-ms 0 2021-03-11T01:04:11.519 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:04:11.520 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:04:12.576 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'list', '--bucket', 'nolznk-49', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:12.587 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'list', '--bucket', 'nolznk-49', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:12.588 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog list --bucket nolznk-49 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:04:12.659 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:04:12.660 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:04:12.839 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2021-03-11T01:04:12.840 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_index_log_trim ... ok 2021-03-11T01:04:12.874 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_creation_time ... ok 2021-03-11T01:04:12.876 INFO:tasks.rgw_multisite_tests:log information for manual testing ... ok 2021-03-11T01:04:12.877 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:12.904 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:12.905 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:04:12.944 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:04:12.944 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:04:13.112 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=76343c7e-8958-4b2a-a007-ca994322980a 2021-03-11T01:04:13.144 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615424641.485395_246.1","last_update":"2021-03-11T01:04:01.485395Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T01:04:13.145 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2021-03-11T01:04:13.146 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:13.147 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:13.148 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:04:13.189 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:04:13.190 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:04:13.403 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615424641.485395_246.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T01:04:01.485395Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T01:04:13.404 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2021-03-11T01:04:13.405 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T01:04:13.405 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:13.406 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:13.406 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:04:13.483 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:04:13.483 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:04:13.740 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615424641.485395_246.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T01:04:01.485395Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T01:04:13.740 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T01:04:13.741 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:13.741 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:13.743 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:04:13.787 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:04:13.788 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:04:13.951 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=76343c7e-8958-4b2a-a007-ca994322980a 2021-03-11T01:04:13.982 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615424641.485395_246.1","last_update":"2021-03-11T01:04:01.485395Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T01:04:13.983 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T01:04:13.984 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:13.985 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:13.987 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:04:14.031 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:04:14.032 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:04:14.290 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615424641.485395_246.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T01:04:01.485395Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T01:04:14.291 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T01:04:14.319 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:14.322 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:14.323 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:04:14.359 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:04:14.360 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:04:14.520 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=76343c7e-8958-4b2a-a007-ca994322980a 2021-03-11T01:04:14.549 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615424654.312265_250.1","last_update":"2021-03-11T01:04:14.312265Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T01:04:14.550 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T01:04:14.551 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:14.552 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:14.555 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:04:14.599 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:04:14.600 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:04:14.851 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615424641.485395_246.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T01:04:01.485395Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T01:04:14.853 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1615424654.312265_250.1 target=1_1615424641.485395_246.1 2021-03-11T01:04:19.855 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:19.863 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:19.864 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:04:19.905 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:04:19.905 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:04:20.164 INFO:teuthology.orchestra.run.smithi124.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"76343c7e-8958-4b2a-a007-ca994322980a","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1615424654.312265_250.1","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"2021-03-11T01:04:14.312265Z","realm_epoch":4}},{"key":1,"val":{"state":1,"marker":"1_1615423619.033309_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2021-03-11T00:46:59.033309Z","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2021-03-11T01:04:20.165 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2021-03-11T01:04:20.474 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:20.475 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:20.475 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:04:20.515 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:04:20.516 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T01:04:20.687 INFO:teuthology.orchestra.run.smithi119.stderr:No --period given, using current period=76343c7e-8958-4b2a-a007-ca994322980a 2021-03-11T01:04:20.715 INFO:teuthology.orchestra.run.smithi119.stdout:[{"marker":"1_1615424654.312265_250.1","last_update":"2021-03-11T01:04:14.312265Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2021-03-11T01:04:20.716 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2021-03-11T01:04:20.716 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:20.717 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2021-03-11T01:04:20.717 DEBUG:teuthology.orchestra.run.smithi124:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2021-03-11T01:04:20.759 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setuser ceph since I am not root 2021-03-11T01:04:20.760 INFO:teuthology.orchestra.run.smithi124.stderr:ignoring --setgroup ceph since I am not root 2021-03-11T12:36:54.145 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2021-03-11T12:36:54.151 DEBUG:teuthology.task.console_log:Killing console logger for smithi119 2021-03-11T12:36:54.152 DEBUG:teuthology.task.console_log:Killing console logger for smithi124 2021-03-11T12:36:54.153 DEBUG:teuthology.exit:Finished running handlers